This U.S. patent application claims priority under 35 U.S.C. § 119 to: Indian Patent Application No. 202121052028, filed on Nov. 12, 2021. The entire contents of the aforementioned application are incorporated herein by reference.
The disclosure herein generally relates to the field of spectral matching and more particularly, to a method and system for spectrum matching for hyperspectral and multispectral data.
In hyperspectral imaging, hyperspectral data is collected with a large number of narrow bands, as opposed to RGB data (which is collected with three broad bands) or multispectral data (which is collected with more than three broad bands). Because of the narrow band data collection in hyperspectral imaging (commonly less than or equal to ten nano meters), the detailed interaction between a material and the solar energy is available. This interaction is recorded as an amount of energy reflected by each wavelength/band (indicating reflectance recorded over a certain range). Hypothetically, reflection and absorption of the solar energy at each wavelength (over a wide wavelength range) by the given material is unique and hence the material is identified by such a unique signature. This is not available in RGB imaging and/or multispectral imaging because of the broad band data collection.
There are several remote sensing techniques for detecting a target material. One of the most common strategies is to record the signature of reference materials and create a library. The signature of the target material (material to be detected, whose signature is available from any source) is then matched with the references in the library. The match can be found for a specific diagnostic absorption or reflectance or for entire spectrum of the target material and reference material. If there are no prominent diagnostic absorptions or reflectance in the spectrum then the entire spectrum is matched. The entire spectrum match is used in hyperspectral image classification (segmentation) procedure. In this classification process a target image pixel is matched with the library for labelling the same. The target image pixel is labelled with the label of the reference material having closest match with the target image pixel. The procedure is repeated for every target image pixel in the target image for segmenting. The spectral matching is also used in deep learning classification as well. A match score between the target image and the reference material is used for measuring reconstruction loss or any other similar loss for optimizing weights of the deep learning network. The loss function corresponding to the above said losses is desired, in addition to loss because of labelling error, to generalize the deep learning model well. The matching is relevant in the other procedures as well wherein exclusive label is not required yet matching between two spectra is essential. For example, the process of clustering is performed using the match score (or similarity measure or distance measure) between two spectra. The pixels are grouped based on the similarity of the pixel, with the pixel (or some hypothetical pixel) representing cluster center.
There are many spectral matching methods such as cosine measure or spectral angle between two spectra, L1 norm corresponds to band wise difference, L2 norm corresponds to Euclidian distance in case the spectra are represented as a real-valued vector. If the spectra are converted to binary representation, then hamming distance is used. Most of these spectral matching methods consider geometric or statistical distances and each one has advantages and disadvantages. For example, if the two spectra are having similar shapes, cosine measure does not account for the amplitude difference in the spectra. For large statistically significant amplitude difference of the spectra, the cosine measure is same. This kind of measures considers the spectrum of the target material as an ‘n’ dimensional vector where ‘n’ equals number of bands. Furthermore, these distances are defined over spectra of equal lengths. There are situations wherein the spectra might not be of equal lengths. There might be different number of bands in the same wavelength range because of differences in spectral-resolution or situation demands matching between multispectral and hyperspectral signatures. For example, when a multispectral spectrum is to be matched with the one in the hyperspectral library.
Embodiments of the present disclosure present technological improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems.
For example, in one embodiment, a method for spectrum matching for hyperspectral and multispectral data is provided. The method includes: receiving a target spectrum of a target object for performing an optimal spectrum matching with a reference spectrum, wherein the target spectrum comprises a first set of primitives and the reference spectrum comprises a second set of primitives represented in a two-dimensional space; transforming each primitive in the first set of primitives of the target spectrum to at least one primitive in the second set of primitives of the reference spectrum using at least one of a first transformation method, a second transformation method and a third transformation method to obtain a set of transformations, wherein each transformation in the set of transformations incurs a transformation cost; obtaining a set of optimal transformations of the first set of primitives to the second set of primitives by optimizing the set of transformations; obtaining a set of global transformation costs by taking the sum of transformation cost corresponding to each optimal transformation of the set of optimal transformations; and obtaining an optimal transformation cost by optimizing the set of global transformation costs using an optimization technique, wherein the optimal transformation cost represents the optimal spectrum matching of the target spectrum with the reference spectrum.
In another aspect, a system for spectrum matching for hyperspectral and multispectral data is provided. The system comprises memory storing instructions; one or more communication interfaces; and one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to receive a target spectrum of a target object for performing an optimal spectrum matching with a reference spectrum, wherein the target spectrum comprises a first set of primitives and the reference spectrum comprises a second set of primitives represented in a two-dimensional space; transform each primitive in the first set of primitives of the target spectrum to at least one primitive in the second set of primitives of the reference spectrum using at least one of a first transformation method, a second transformation method and a third transformation method to obtain a set of transformations, wherein each transformation in the set of transformations incurs a transformation cost; obtain a set of optimal transformations of the first set of primitives to the second set of primitives by optimizing the set of transformations; obtain a set of global transformation costs by taking the sum of transformation cost corresponding to each optimal transformation of the set of optimal transformations; and obtain an optimal transformation cost by optimizing the set of global transformation costs using an optimization technique, wherein the optimal transformation cost represents the optimal spectrum matching of the target spectrum with the reference spectrum.
In an embodiment, wherein the target spectrum and the reference spectrum are at least one of a hyperspectral spectrum or a multispectral spectrum.
In an embodiment, wherein the first transformation method corresponds to an edit distance based method based on a distance measured between the first set of primitives and the second set of primitives.
In an embodiment, wherein the second transformation method corresponds to a Hungarian based method based on x and y dimension distance measure between the first set of primitives and the second set of primitives.
In an embodiment, wherein the third transformation method corresponds to a piece-wise angular distance method based on angular difference and length difference between the first set of primitives and the second set of primitives.
In an embodiment, wherein the transformation cost corresponds to one or more of (i) translation of a primitive in x and y dimensions (ii) deletion of the primitive and (iii) insertion of the primitive.
In yet another aspect, a non-transitory computer readable medium for spectrum matching for hyperspectral and multispectral data is provided by receiving a target spectrum of a target object for performing an optimal spectrum matching with a reference spectrum, wherein the target spectrum comprises a first set of primitives and the reference spectrum comprises a second set of primitives represented in a two-dimensional space; transforming each primitive in the first set of primitives of the target spectrum to at least one primitive in the second set of primitives of the reference spectrum using at least one of a first transformation method, a second transformation method and a third transformation method to obtain a set of transformations, wherein each transformation in the set of transformations incurs a transformation cost; obtaining a set of optimal transformations of the first set of primitives to the second set of primitives by optimizing the set of transformations; obtaining a set of global transformation costs by taking the sum of transformation cost corresponding to each optimal transformation of the set of optimal transformations; and obtaining an optimal transformation cost by optimizing the set of global transformation costs using an optimization technique, wherein the optimal transformation cost represents the optimal spectrum matching of the target spectrum with the reference spectrum.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the disclosed principles:
Exemplary embodiments are described with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Wherever convenient, the same reference numbers are used throughout the drawings to refer to the same or like parts. While examples and features of disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments.
The embodiments herein provide a method and system for spectrum matching for hyperspectral and multispectral data. The disclosed method explains spectrum matching as a measurement of transformation required for converting a target spectrum to a reference spectrum in a library or vice versa. Each primitive in a set of primitives of the target spectrum is transformed to the primitive of the reference spectrum. Each transformation has a cost associated with it. The total cost of transformation is taken as a distance between the two spectra. The disclosed method performs spectral matching based on a distance between the two spectra which reflect the shape difference between the two spectra in two-dimensional space (wavelength and reflectance).
Referring now to the drawings, and more particularly to
The I/O interface (s) 106 may include a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like and can facilitate multiple communications within a wide variety of networks and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O interface(s) can include one or more ports for connecting a number of devices to one another or to another server.
The memory 104 may include any computer-readable medium known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes.
In an embodiment of the present disclosure, the one or more processors 102 are configured to receive at step 202, a target spectrum of a target object for performing an optimal spectrum matching with a reference spectrum, wherein the target spectrum comprises a first set of primitives and the reference spectrum comprises a second set of primitives represented in a two-dimensional space. The x-axis and y-axis in two-dimensional space correspond to wavelength and reflectance respectively. In an embodiment, the target spectrum can be hyperspectral or multispectral spectrum and the reference spectrum is hyperspectral spectrum, however in some cases the reference spectrum can be multispectral spectrum also. The target spectrum can be received as a single spectrum or as a collection of spectra (collection of spectra in image or non-image format). The set of primitives of the target spectrum and reference spectrum can be curves or a set of points.
In an embodiment of the present disclosure, the one or more processors 102 are configured to transform at step 204, each primitive in the first set of primitives of the target spectrum to at least one primitive in the second set of primitives of the reference spectrum using at least one of a first transformation method, a second transformation method and a third transformation method to obtain a set of transformations, wherein each transformation in the set of transformations incurs a transformation cost. In an embodiment, the transformation method is chosen based on sensitivity measure of the three transformation methods. In general, sensitivity is a measure of changes in the distance per unit perturbations in an original spectrum. White noise is added to the original spectra and then a match is measured between the original and changed spectra. This distance indicates the increase in the distance because of the disturbances added in the original spectrum (as the distance with self is zero). Further, the distance is divided by the total perturbations added. Higher the sensitivity better is the method for calculating match score. Table 1 provides a comparative assessment of sensitivity of various prior distance measures with the disclosed methods for hyperspectral signatures.
In another embodiment, the method for transformation for spectral matching can also be selected based on other attributes as well such as computational complexity of the method, domain knowledge (encoding the expert knowledge in form of rules and using rule based or case based system for selecting the method), meta empirical or analytical method or ensemble methods (voting or taking weighted summation of distances from all methods and weightage assigned to outcome of each method wherein weightage is assigned by experts or calculated automatically by internal properties of the method). The first transformation method corresponds to edit distance method, the second transformation method corresponds to Hungarian method and the third transformation method corresponds to piece-wise angle and length distance method. The explanation and algorithm for the first, second and third transformation methods are provided further after step 206. The steps of the algorithm for first, second and third transformation methods provided further includes the step 206 of the method 200.
In an embodiment of the present disclosure, the one or more processors 102 are configured to obtain at step 206, a set of optimal transformations of the first set of primitives to the second set of primitives by optimizing the set of transformations.
According to an embodiment of the disclosure, the steps for the edit distance method are provided below:
1.Create a first matrix for pairs from target spectrum primitives and reference spectrum primitives.
2. Starting from left to right and starting at first primitive of the target spectrum calculate the three distances with the mapped/paired primitive on reference spectrum. The three distance operations are insertion, deletion and substitution. If the target spectrum and the reference spectrum are of equal length, insertion and deletion operation is not needed. Minimum of the three distances is chosen for finding the total transformation cost and cost associated with the operation is added.
3. Repeat the operation for all the paired primitives and sum up the cost for selected operations for each paired transformation to obtain the total transformation cost.
According to an embodiment of the disclosure, the steps for the Hungarian based method are provided below:
1. Create a second matrix of pairs of the primitives from the target spectrum and the reference spectrum.
2. In this case the primitive from target spectrum can be mapped or transformed to any other primitive in the reference spectrum. A cost is associate for each such transformation of paired primitives.
3. Calculate the cost of transformation for each pair. The cost for a transformation for a pair is: weighted x and y scalar difference (Δx and Δy). By end of this step a matrix with all possible transformation and their cost would be ready. Thus, for example, assuming equal length spectrums of n bands there would be n×n pairs of assignments.
According to an embodiment of the disclosure, the steps for the piece-wise angle and length distance method are given below:
1. Calculate an angular difference and a length difference between corresponding arcs of the target spectrum and the reference spectrum to generate an arc score wherein the target spectrum and the reference spectrum are represented by two dimensions wavelength and reflectance values.
2. Repeat the step 1 for every corresponding arc in the target and the reference spectrum.
3. Add the arc scores corresponding to all arcs to get the final matching score.
In an embodiment another variant for the piece-wise angle and length method is provided as follows. The n dimensional vector can be viewed from n−1 two dimensional directions, three dimensional directions and so on. For example, a point in XYZ dimensions can be orthographically projected to XY, YZ, or XZ plane or can be viewed from the direction which is perpendicular to the plane. Such a pair of dimensions in sequence can represent vector in subspace. The cosine for such a projected vector between the reference and the target spectrum is calculated for each two dimensions in sequence and then added up to give the total distance. For example, if a spectrum A has 8 bands named as band a1, a2, a3, a4 and so on. And spectrum B has 8 bands b1, b2, b3, b4 and so on. Then vector in subspace 1-2 for target and reference is considered. Its cosine is calculated that is cosine between vector (a1, b1) and vector (b1, b2). Same procedure is repeated for all the ordered pairs such as (a2,a3) and (b2,b2) and so on and the cosine in for each such pair is aggregated to give the total similarity between the two spectra.
The pseudo code for the edit distance method is provided below Inputs:
Band_Wavelengths, Spectrum A, Spectrum B
dist: =0
dp_matrix:=empty_matrix(shape=(num_of_bands, num_of_bands))
The pseudo code for the Hungarian method is provided below
Band_Wavelengths, Spectrum A, Spectrum B, param a, param A
dist:=0
dist_matrix=empty_matrix(shape=(num_of_bands, num_of_bands))
The pseudo code for the piece-wise angle and length method is provided below
Band_Wavelengths, Spectrum A, Spectrum B, Param A
dist:=0
In an embodiment of the present disclosure, the one or more processors 102 are configured to obtain at step 208, a set of global transformation costs by taking the sum of transformation cost corresponding to each optimal transformation of the set of optimal transformations. The set of global transformation cost is the sum of (costs for) optimal pairs of primitives of the target spectrum and the reference spectrum. The best pair of transformation for each primitive cannot be decided using the local minima.
In an embodiment of the present disclosure, the one or more processors 102 are configured to obtain at step 210, an optimal transformation cost by optimizing the set of global transformation costs using an optimization technique, wherein the optimal transformation cost represents the optimal spectrum matching of the target spectrum with the reference spectrum. Optimization is performed using any suitable optimization method such as dynamic programming method or the Hungarian method for calculating the best matching score between the target spectrum and the reference spectrum. Higher the matching score better is the discrimination between the two spectra. The score can be inversed to convert it to a similarity score where high score indicates a good match.
EXPERIMENTAL RESULTS: The performance of the disclosed method on spectral databases available in public domain are analyzed. Publicly available spectral libraries such as ASTER, splib07, and splib Tarnag are used for analyzing the performance of the disclosed method. Spectra of most representative materials in each Visible Spectroscopy (VIS) categories were selected. For example, three number of samples from vegetation, impervious surfaces (such as concrete, asphalt etc.), and soil are selected. The code of the materials as per the library nomenclature is provided in the table for easy reference. The original spectral spectra were resampled to a common wavelength range by a simple interpolation. The new range was different for spib07 and ASTER library. It is <from-to-interval> for splib07 and <from-to-interval> for ASTER. This is not essential as the disclosed method do not require two vectors of same length for distance calculation. However, it is essential for comparative assent with the other methods. Table 2, Table 3, and Table 4 shows a comparison of distances of the edit distance based method, the Hungarian based method and the piece-wise angular distance method respectively for various spectral signatures.
The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.
It is to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g., any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g., hardware means like e.g., an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g., an ASIC and an FPGA, or at least one microprocessor and at least one memory with software processing components located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g., using a plurality of CPUs.
The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various components described herein may be implemented in other components or combinations of other components. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
The illustrated steps are set out to explain the exemplary embodiments shown, and it should be anticipated that ongoing technological development will change the manner in which particular functions are performed. These examples are presented herein for purposes of illustration, and not limitation. Further, the boundaries of the functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. Such alternatives fall within the scope of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” and other similar forms are intended to be equivalent in meaning and be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.
Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the present disclosure. A computer-readable storage medium refers to any type of physical memory on which information or data readable by a processor may be stored. Thus, a computer-readable storage medium may store instructions for execution by one or more processors, including instructions for causing the processor(s) to perform steps or stages consistent with the embodiments described herein. The term “computer-readable medium” should be understood to include tangible items and exclude carrier waves and transient signals, i.e., be non-transitory. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical storage media.
It is intended that the disclosure and examples be considered as exemplary only, with a true scope of disclosed embodiments being indicated by the following claims.
Number | Date | Country | Kind |
---|---|---|---|
202121052028 | Nov 2021 | IN | national |