The present technology relates to systems and methods for correcting imaging data collected by imaging sensors, and in particular to methods and calibration systems that use calibration arrays that emit a test pattern.
A variety of test patterns are known for use in calibrating standard optical sensors. For example, physical or digital cards having a known pattern, whether of black, white, patterns of black and white, or patterns of color, have all been used as a basis for adjusting chroma and tint with respect to cameras and visual broadcast receivers. Most modern test patterns for optical sensors include a known set of color bars that produce a characteristic pattern of “dot landings” on a vectorscope, so that precise adjustments to equipment can be made.
Normally, a test pattern is used to calibrate equipment before the equipment is used for its intended purpose. In such instances, the equipment is set up, the sensor is aimed at the test pattern having known characteristics, and an operator adjusts the equipment settings so that the data output by the sensor accurately describes the known characteristics of the test pattern. For example, in television programming, a test pattern can be used in a studio, or in the on-location environment of an intended broadcast, just prior to conducting the broadcast, in order to calibrate the cameras by adjusting their output adjust the image quality of the cameras, which can be affected by environmental conditions such as the lighting conditions. In another example, monitoring equipment that is going to be used to monitor remote locations, such as equipment used on an airplane or a satellite, is normally calibrated using test patterns in lab or local field conditions to adjust the equipment settings and output before the equipment is used.
Calibration methods and systems of the present technology include a calibration array that can be controlled to generate a test pattern having geometric, temporal and electromagnetic characteristics.
In one aspect, a method of correcting imaging data received from a sensor is provided. The method can include placing a calibration system into an environment comparable to an environment from which imaging data is to be collected by a sensor during a data collection time period. The calibration system can be configured to emit a test pattern having geometric, temporal, and electromagnetic characteristics, the calibration system having at least one emission point, each emission point including at least one emission source that emits in a region of the electromagnetic spectrum. The calibration system can then emit the test pattern having the geometric, temporal, and electromagnetic characteristics with the calibration array during the data collection time period. The imaging data can be collected from the calibration system with a sensor during the data collection time period. The method can further include comparing the imaging data collected from the calibration system by the sensor to the geometric, temporal, and electromagnetic characteristics of the test pattern to determine an imaging error factor based on differences between the hyperspectral imaging data collected from the calibration system by the sensor and the geometric, temporal, and electromagnetic characteristics of the test pattern, and correcting the imaging data by the error factor.
In another aspect, a calibration system is provided that can be placed into an environment comparable to an environment from which imaging data is to be collected by a sensor and used for correcting imaging data received from the sensor. The calibration system can include a calibration array, a control unit operatively connected to the calibration array, and a transmitter operatively connected to the control unit. The calibration array can emit a test pattern having geometric, temporal, and electromagnetic characteristics. The calibration array can include a plurality of emission points, each emission point including at least one emission source that emits in a region of the electromagnetic spectrum. The control unit can provide test pattern commands to the calibration array in order to cause the calibration array to emit the test pattern. The transmitter can transmit data from the calibration system to a remote location.
Specific examples have been chosen for purposes of illustration and description, and are shown in the accompanying drawings, forming a part of the specification.
Imaging sensors can be used to collect imaging data and create images of a monitored environment. A monitored environment can be any environment monitored by an imaging sensor, and, depending upon the application, can include any physical location or geographic area. In some non-limiting examples, imaging sensors can be carried by an aircraft or satellite. However, environmental conditions and atmospheric variables, such as temperature, airborne dust, water vapor, ozone, and atmospheric attenuation, can distort imaging data collected by an imaging sensor, so that even if the imaging sensor is properly calibrated the output of the sensor will not provide an accurate image based on the collected imaging data.
Methods and systems of the present technology can be used for correcting imaging data received from a sensor to compensate for such environmental effects by using test patterns having geometric, temporal, and electromagnetic characteristics.
One type of imaging sensor that can be used with the methods and systems of the present technology is a trans-spectral imaging sensor. Trans-spectral imaging sensors can be used to create images of a monitored environment utilizing light from a plurality of regions of the electromagnetic spectrum, ranging from ultraviolet (UV) to far infrared, including for example visible light, near infrared (NIR), short wave infrared (SWIR), medium wave infrared (MIR), and long wave infrared (LWIR). In particular, as used herein, the term trans-spectral refers to the use of light from at least two regions of the electromagnetic spectrum. Trans-spectral imaging sensors collect trans-spectral imaging data to form images of a monitored environment.
In some examples, such as applications that use trans-spectral imaging sensors, the test patterns can be trans-spectral test patterns. As used herein, the term trans-spectral test pattern refers to a test pattern that includes emissions in two or more regions of the electromagnetic spectrum. For example, a trans-spectral test pattern can include a first emission in a region of the electromagnetic spectrum selected from the group consisting of ultraviolet (UV), visible light, near infrared (NIR)(wavelength of 0.75 μm to 1.4 μm), short wave infrared (SWIR)(wavelength of 1.4 μm to 3 μm), medium wave infrared (MIR)(wavelength of 3 μm to 8 μm), long wave infrared (LWIR) (wavelength of 8 μm to 15 μm), and far infrared (FIR)(wavelength above 15 μm). A trans-spectral test pattern can also include a second emission in a region of the electromagnetic spectrum selected from the same group, but that is different from the region of the electromagnetic spectrum of the first emission.
With reference to
Further, the data collection time period can have a duration and a frequency. The duration of the data collection time period can be any desired duration, ranging from a fraction of a second up to a number of years, including seconds, minutes, days, weeks, or months. The frequency of the data collection time period can be a single instance, or can be periodic. Thus, in some examples, the data collection time period could be a single instance in which trans-spectral imaging data is collected for a fraction of a second, or a continuous period in which trans-spectral imaging data is collected for a plurality of minutes or hours, or even a periodic occurrence in which trans-spectral imaging data is collected for a plurality of seconds or minutes each day.
Once the calibration system 100 is placed, the calibration array 102 can emit the trans-spectral test pattern at some point during the data collection time period, and the trans-spectral sensor 112 can collect trans-spectral imaging data. The trans-spectral sensor can have a field of view from which it collects the trans-spectral imaging data 110. The field of view can cover an area of the environment in which the calibration system 100 has been placed. The area covered by the field of view can include at least the area occupied by the calibration system 100, and can include additional area around the calibration system 100. The additional area can span any distance, such as inches, feet, kilometers, or miles.
The trans-spectral test pattern emitted by the calibration array 102 can have geometric, temporal, and electromagnetic characteristics. For example, referring to
Referring to
The trans-spectral test pattern can be generated by activating at least a first emission source 106 at a first emission point 104 and a second emission source 106 at the first or any other emission point 104, where the first emission source 106 emits with a first electromagnetic signature and the second emission source 106 emits with a second electromagnetic signature that is different from the first signature. At least one emission source 106 at any plurality of emission points 104 can be activated to form a trans-spectral test pattern.
Additionally, the trans-spectral test pattern can be varied by altering or varying the number of emission sources 106 or emission points 104 that are activated at or during any given data collection time. Trans-spectral test patterns can be varied with respect to several factors, including geometric configuration, time, emission wavelength, and intensity. A trans-spectral test pattern can be arranged to provide a pattern having areas with and without light sources. Some trans-spectral test patterns can be designed for narrow bands of operation, and can utilize emission sources with output that is limited to a desired narrow band of the spectrum. Alternatively, trans-spectral test patterns can include a wide spectrum of emissions within the electromagnetic spectrum.
The electromagnetic characteristics of the trans-spectral test pattern can include the regions of the electromagnetic spectrum in which each activated emission source 106 emits, including the particular frequency or range of frequencies at which each activated emission source 106 emits.
The geometric characteristics of the trans-spectral test pattern can include the location of each emission point 104 at which an emission source 106 is activated to generate the trans-spectral test pattern. The geometric characteristics of the trans-spectral test pattern can also include a shape that is formed by the activated emission sources 106 at the emission points 104 included in the trans-spectral test pattern. The shape can be any regular or irregular shape, including but not limited to a line, a square, a rectangle, a circle, an oval, or a grid.
The temporal characteristics of the trans-spectral test pattern can include the clock time at which the test pattern is emitted, the duration of time during which the test pattern is emitted, and any timing of variations in the test pattern emissions. For example, the system could emit a trans-spectral test pattern that includes an emission from a first emission source in a first region of the electromagnetic spectrum at a first time, and an emission from a second emission source in a second region of the electromagnetic spectrum at a second time.
Referring to
The control unit 116 can transmit test pattern commands from the control unit to the calibration array in order to cause the calibration array to emit the trans-spectral test pattern. In one such an example, the control unit 116 can be pre-programmed prior to being placed in the environment from which trans-spectral imaging data is to be collected to cause the calibration array to generate a certain trans-spectral test pattern at a certain data collection time. In another example, the calibration system 100 can include a communications receiver 118 operatively connected to the control unit 116 that receives test pattern commands from a remote location and provides the test pattern commands to the control unit 116. Alternatively, control unit 116 can be pre-programmed to actuate calibration array 102 at and for specific times (e.g., when a satellite passes over).
Referring back to
The error factor is based upon the illumination of the focal plane array of sensor 112 by emission sources 106 based upon relative location and orientation of the calibration array 102 to the sensor 112, and the field of view of each of the individual pixels of sensor 112. Under ideal atmospheric conditions, an estimate of the power of each emission source 106 can be determined for each focal plane array pixel of sensor 112. Then actual measurements are collected from the sensor 112 when the array 102 is emitting and in the sensor field of view. The difference between the ideal image and the actual image of the array 102 provides a correction or error factor to compensate for non-ideal atmospheric propagation. Since the emission sources 106 can be modulated for operation individually and in groups across the spectrum, it is possible that the error factors may correct for limited cross-pixel image blur from atmospheric turbulence or density variations and minor imager optics lens aberrations.
The imaging data 110 collected by the sensor 112 can then be analyzed based on the error factor. In this manner, the effect of distortions of imaging data, which can be caused by weather conditions, lighting conditions, and atmospheric attenuation, among other things, can be determined and accounted for when analyzing the collected hyperspectral imaging data. The error factor determined from the pixels of the sensor 102 focal plane in which array 102 is imaged can now be used to compensate image data across the entire focal plane array and collected images in the comparable environment of the array 102. The error factor data is added logarithmically to each pixel for each band of electromagnetic spectrum in which the sensor 102 operates.
As illustrated in
In the illustrated example, the calibration system 100 can include communication system such as a transmitter 122 operatively connected to the control unit 116 that transmits data from the calibration system to a remote location. Alternatively, the communication system may include a memory device or may be hard-wired. As used herein, a remote location should be understood to mean any location that is not internal to the calibration system 100, but is instead outside of and separate from the calibration system 100. A remote location can be located at any distance from the calibration system 100, and can be within or outside of the environment from which imaging data is to be collected. In this manner, potentially data can be provided from the calibration system 100 to users operating the imaging sensor and analyzing the imaging data received from the imaging sensor.
Another component that the calibration system 100 can include is built in test equipment 124, which can be operatively connected to the control unit 116. The built in test equipment 124 can collect test pattern data while the calibration array is emitting the trans-spectral test pattern. The test pattern data collected by the built in test equipment 124 can include the state of each emission point 104, or of each emission source 106, of the calibration array 102. The state of an emission point 104 can include information regarding whether any emission source 106 was activated, as well as the electromagnetic characteristics of any activated emission source 106, the time at which any emission source 106 was activated, and/or the duration for which any emission source 106 was activated. The test pattern data can also include the geometric characteristics of the test pattern actually generated by the calibration array 102. The test pattern data collected by the built in test equipment 124 can be transmitted to the remote location by the transmitter 122. The test pattern data collected by the built in test equipment 124 can be compared to the geometric, temporal, and electromagnetic characteristics of the test pattern to determine a test pattern error factor based on any differences between the test pattern data and the geometric characteristics and electromagnetic characteristics of the test pattern, and the imaging error factor can be adjusted based on the test pattern error factor. Alternatively, the built in test equipment output can become the calibration array emission data set used to project the calibration array emissions into the image sensor focal plane array pixels.
A calibration system 100 can also include an environmental monitor 126, which can be operatively connected to the control unit 116. The environmental monitor 126 can collect environmental data regarding the environment in which the calibration system is placed. Such environmental data can include, for example, temperature, wind velocity, precipitation, and/or barometric pressure. The environmental monitor 126 can also collect environmental data regarding the internal or surface environment of the calibration array. For example, when the trans-spectral test pattern includes LWIR emissions, it may be desirable to collect environmental data regarding the surface temperature of the calibration array 102 and the temperature of the emission source 106 emitting the LWIR signal. Environmental data collected by the environmental monitor 126 can be included in the data transmitted to the remote location by the transmitter 122.
A calibration system 100 can also include a GPS receiver 128, which can be operatively connected to the control unit 116. The GPS receiver 128 can receive global positioning signals and generate positioning data, including for example the longitude and latitude at which the calibration system 100 is placed, which can be included in the data transmitted to the remote location by the transmitter 122.
Alternatively or in addition to the GPS receiver 128, a calibration system 100 can include a clock 130, which can be operatively connected to the control unit 116. The clock 130 can provide time data, which can be included in the data transmitted to the remote location by the transmitter 122. Time data can include, for example, the time at which the calibration system 100 was placed, times at which the calibration system 100 generates a trans-spectral test pattern, and/or the duration for which the calibration system 100 generates a trans-spectral test pattern. The clock 130 can also be employed to control the actuation of calibration array 102.
The calibration system 100 can also include a power source 132, which can be used to power the components of the calibration system 100. The power source 132 can include a primary power source 134 and a secondary power source 136. The primary power source can include at least one rechargeable battery. The secondary power source can include a power generator that generates power from an environmental stimulus, such as a solar panel that generates power from sunlight, or a windmill that generates power from the wind. Alternatively, the secondary power source can generate electricity from any other energy source. Power generated by the secondary power source can be used to recharge the primary power source, or to provide power directly to the components of the calibration system 100.
The calibration system 100 can also include an orientation sensor 146, which can be operatively connected to the control unit 116. The orientation sensor 146 can measure and collect orientation data regarding the orientation of the calibration system 100, including directional orientation, which can be measured in three dimensional orientation terms such as yaw, pitch, and roll.
Referring to
When the emissions sources 106 of the present technology include fiber optics, a plurality of fiber optics can be provided, where each fiber optic is an emission source 106 at an emission point 102. The fiber optics can be used to carry a signal from a signal source 140, either coherent or non-coherent, to each emission point 104 having a fiber optic emission source 106. Optical phase shifters 142 can also be included, and can be connected to the single source and each fiber optic emission source 106. The phase shifters 142 can be used to cause the signal from the signal source 140 to be phase shifted signal across the calibration array 102.
From the foregoing, it will be appreciated that although specific examples have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit or scope of this disclosure. It is therefore intended that the foregoing detailed description be regarded as illustrative rather than limiting, and that it be understood that it is the following claims, including all equivalents, that are intended to particularly point out and distinctly claim the claimed subject matter.
This application is a Continuation Application based on U.S. patent application Ser. No. 14/558,416, filed Dec. 2, 2014, which issued as U.S. Pat. No. 9,118,811 on Aug. 25, 2015. U.S. Pat. No. 9,118,881 is based on and derives the benefit of the filing date of U.S. patent application Ser. No. 13/887,407, filed May 6, 2013, which issued as U.S. Pat. No. 8,928,758 on Jan. 6, 2015. U.S. Pat. No. 8,928,758 is based on and derives the benefit of the filing date of U.S. Provisional Patent Applications No. 61/661,161, filed Jun. 18, 2012 and No. 61/679,090, filed Aug. 3, 2012. The entire content of all of these applications are herein incorporated by reference in their entireties.
Number | Name | Date | Kind |
---|---|---|---|
4343021 | Frame | Aug 1982 | A |
5020116 | Macaulay | May 1991 | A |
5563420 | Sullivan | Oct 1996 | A |
5572444 | Lentz | Nov 1996 | A |
5650844 | Aoki | Jul 1997 | A |
5734158 | Nagashima | Mar 1998 | A |
6072603 | Parks | Jun 2000 | A |
6509956 | Kobayashi | Jan 2003 | B2 |
6542185 | Bogardus | Apr 2003 | B1 |
7200497 | Wang | Apr 2007 | B2 |
7515094 | Keller, III | Apr 2009 | B2 |
7539563 | Yang | May 2009 | B2 |
7932917 | Han | Apr 2011 | B2 |
8189957 | Tan | May 2012 | B2 |
8300103 | He | Oct 2012 | B2 |
8629909 | Askey | Jan 2014 | B2 |
20030007077 | Maggi | Jan 2003 | A1 |
20050243286 | Kobayashi | Nov 2005 | A1 |
20060138488 | Kim | Jun 2006 | A1 |
20100112469 | Miyaharu | May 2010 | A1 |
20100125356 | Shkolnik | May 2010 | A1 |
20130002443 | Breed | Jan 2013 | A1 |
20130088589 | Negoro | Apr 2013 | A1 |
20140098336 | Mo | Apr 2014 | A1 |
Number | Date | Country |
---|---|---|
20020296709 | Oct 2002 | JP |
Entry |
---|
Galilea et al., “Calibration of a High-Accuracy 3-D Measurement Sensor Based on Laser Beam and CMOS Camera.” IEEE Transactions on Instrumentation and Measurement, 2009, pp. 3341-3346. |
Woodward Woodward et al., “Hyperspectral Imager Characterization and Calibration.” IEEE International Geoscience and Remote Sensing Symposium, 2009, pp. 77-80. |
U.S. Appl. No. 13/887,407. |
U.S. Appl. No. 14/558,416. |
Number | Date | Country | |
---|---|---|---|
20150324986 A1 | Nov 2015 | US |
Number | Date | Country | |
---|---|---|---|
61661161 | Jun 2012 | US | |
61679090 | Aug 2012 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 14558416 | Dec 2014 | US |
Child | 14801711 | US | |
Parent | 13887407 | May 2013 | US |
Child | 14558416 | US |