Patent Application
20190249956
This application claims the priority, under 35 U.S.C. § 119, of German application DE 10 2018 001 076.8, filed Feb. 10, 2018; the prior application is herewith incorporated by reference in its entirety.
The invention relates to a method for determining characteristic-curve correction factors of a matrix detector which images in the infrared spectral range.
Vehicles under automated control may comprise a matrix detector, on which a region of the surrounding object scene is imaged. The image contents are evaluated, and the vehicle is controlled on the basis of the evaluation results, for example guided around an identified obstacle or guided to an identified target. However, a problem arising in infrared images, in particular, is that the individual detector elements of the matrix detector have significant image deviations relative to one another due to the manufacturing process, and so clear structures are visible, even in an image without imaged objects. Such image artefacts can lead to an incorrect control of the vehicle and should therefore be corrected to the best possible extent.
Such a correction of infrared detectors is known as a so-called nonuniformity correction/compensation (NUC). However, depending on how the matrix detector is produced, significant image artefacts remain even after such a correction, the image artefacts interfering with a reliable control of the vehicle.
Therefore, it is an object of the invention to specify an improved method for correcting characteristics of a matrix detector that images in the infrared spectral range.
This object is achieved by the method set forth at the outset, in which, according to the invention, an area of homogeneous temperature is recorded at two different temperatures by the matrix detector, there being two images with different integration times for each temperature, and a signal gradient over the integration time is established for each of the pixels from the four pixel values at the two temperatures in each case and the gain is established from the difference of the signal gradients and characteristic-curve correction factors for the gain are stored.
A matrix detector contains a multiplicity of detector elements that are arranged in two dimensions in relation to one another, the detector elements each being able to output a pixel value as a signal, the pixel value increasing over the integration time and also being referred to as signal below. The signals of a plurality of detector elements can also be combined to form a single pixel value, which is why a distinction is made below between detector element and pixel. The signal depends, firstly, on the photon flux Φ striking a detector element and, secondly, on the detector element properties which determine the conversion of the photon flux into the signal.
The signal can be composed from three components: the signal offset ω, also simply referred to as offset, the gain γ and the dark current θ. While the signal offset ω is independent of photon flux Φ and integration time t, the signal increases in integration-time-dependent fashion as a result of the gain γ and the dark current θ, with, however, only the gain γ depending on the photon flux Φ.
In summary, the following relation arises:
s
i=ωi+t(γif(Φi)+θi)
where
si: signal of the detector element i,
Φi: photon flux,
ωi: signal offset of the detector element i,
γi: gain of the detector element I, and
θi: dark current of the detector i.
In order now to arrive at a corrected signal S, the measured value should be corrected by the characteristic-curve correction factors for the signal offset, gain and dark current such that the following relation arises:
S
i=Ωi+t(Γif(Φi)+Θi)
where
Si: corrected signal of the detector element i,
Ωi: characteristic-curve correction factor for the signal offset of the detector element i,
Γi: characteristic-curve correction factor for the gain of the detector element I, and
Θi: characteristic-curve correction factor for the dark current of the detector element i.
The gain and the characteristic-curve correction factor for the gain are equivalent terms since the knowledge about the gain for a pixel contains the knowledge about the characteristic-curve correction factor of the gain for these pixels, which emerges from the ratio of the gain of the pixel to a reference value, which can be the same for all pixels. Therefore, the terms of gain and characteristic-curve correction factor for the gain are used synonymously below. The same applies to the signal offset and the dark current. Moreover, the index i is dispensed with below for reasons of clarity, with the characteristic-curve correction factors expediently being established for each pixel. If a pixel is present for each detector element, the characteristic-curve correction factors are expediently established for each detector element.
While it is relatively simple to establish the signal offset or the characteristic-curve correction factors thereof for each pixel, determining the gain is more complicated. However, the invention provides a simple method for pixel-by-pixel determination of the characteristic-curve correction factors of the gain such that the signal of the pixels or of the detector elements can have a very good correlation therewith. As a result, the overall image becomes substantially more homogeneous, particularly if, optionally, the signal offset is additionally corrected, and the vehicle can be controlled more reliably.
The area with a homogeneous temperature emits at the same temperature or with the same temperature distribution in each recorded region; here, use can be made of a spectrally homogeneous blackbody emitter, the temperature of which lies in the infrared spectral range. All signals of the pixels or detector elements should be the same when recording the area with a homogeneous temperature. In reality, this is not the case. In this respect, it is possible to use deviations from a common reference value, for example the mean value of the gain over all pixels, for the purposes of determining the characteristic-curve correction factors pixel-by-pixel. Deviations of the gradients or gradient differences from the mean value of the gradients or differences can be used as characteristic-curve correction factors for the gain. Expediently, the characteristic-curve correction factors for the gain are determined in a building, for example in a laboratory, and prior to deployment of the matrix detector in a vehicle. The characteristic-curve correction factors for the signal offset may also only be determined during a deployment of the detector in the vehicle.
The integration times are different for the two images of the same temperature, wherein use can be made of the same integration time pairs for the different temperatures. It is just as easily possible to choose three or four different integration times, and so each of the four images was recorded with an individual integration time.
Two signals with different strengths are available for each temperature, it being possible to place a linear function there through depending on the integration time. In this case, the signal gradient is constant. Gradient difference can be established, for example in the form of a value, from the two different gradients for the two temperatures. A reference value for all pixels can be established in the same way, for example a mean value of all signal gradients or gradient differences, a median or any other suitable value. The difference between the pixel-individual gradient difference and the gradient reference value can be stored as a characteristic-curve correction factor. Expediently, storing is implemented by saving to a memory which is expediently accessible to image processing electronics for correcting the images recorded by the matrix detector.
As mentioned, in an advantageous embodiment of the invention, it is expedient for the characteristic-curve correction factors for the signal offset for each of the pixels to be established from at least two pixel values assigned to one of the temperatures. By way of example, the signal value emerging from the gradient at the integration time value of t=0 is taken. Expediently, the signal offset is subtracted from the pixel values and the gain is established from the pixel values remaining thus.
The temperature homogeneity of the imaged area of the blackbody emitter is of substantial importance to the quality of the signal correction. It is therefore expedient for the blackbody emitter or the area with a homogeneous temperature to be imaged in a building in order to obtain the best possible control over the imaged area. Accordingly, it is advantageous if the characteristic-curve correction factors for the gain are determined while the matrix detector is situated in a building. The characteristic-curve correction factors for the signal offset, too, can be determined in the building. However, the signal offset may be subject to significant migration over a period of days, weeks or months, and so it deviates significantly from the laboratory values during a subsequent deployment. However, since the signal offset is dependent on neither the temperature nor the integration time, it can also be determined during a subsequent deployment of the matrix detector in a vehicle, for example in a guided missile. The characteristic curves of the detector elements assigned to the pixels can then be corrected during the deployment using the characteristic-curve correction factors for the signal offset. Naturally, it is also possible to already establish and save the signal offset under laboratory conditions. Then, an offset correction can be established during deployment, the offset correction complementing or overwriting the laboratory-related characteristic-curve correction factors during the deployment.
In the case of a temporally continuous radiant flux of the area with a homogeneous temperature, the images with different integration times can be recorded in temporal proximity of one another to a greater or lesser extent. However, should the signal offset be established during the deployment, an object scene of the surroundings with presumably greater fluctuations will be imaged on the matrix detector. In this situation, in particular, it is very advantageous if the two images with different integration times are recorded immediately in succession. Here, immediately can mean that no further images, or at most three further images, recorded by the matrix detector lie between the two images. By way of example, alternating images with different integration times are continuously recorded by the matrix detector.
An even more in-depth image correction can be achieved if a dark current is determined for each of the pixels using at least the pixel values assigned to one of the temperatures. The characteristic curves of the detector elements assigned to the pixels can now be corrected using the characteristic-curve correction factors of the dark current.
The dark current, too, is dependent on the integration time, and so the signal gradients are formed, first, from the constant component of the dark current and, second, from a photon-flux-dependent or temperature-dependent component of the gain. Only the gain reacts to changes in the scene. A problem here is that although there is a theoretic dependence on the blackbody temperature for the photon flux, so many variables are included in the overall detector response or the signal of the individual detector elements that there is no simple model for the dependence of the signal as a function of the photon flux. All that is certain is the monotonic property of the temperature, with the signal thus increasing with increasing temperature. Accordingly, it is difficult to separate gain from dark current. However, on account of its insensitivity to temperature changes, the dark current only plays the role of an offset in the gradient images; i.e., it is constant with temperature. In this respect, using the gradients allows the dark current to be subtracted as a constant that is only dependent on the individual pixels.
In this respect, one of the signal gradients or a summation of both signal gradients can be used as dark current. In this context, a summation is a variable containing the sum or the difference of the two signal gradients, for example the mean value of the two signal gradients. In this respect, the deviation of the mean value of the two signal gradients of a pixel or detector element from the mean value of the signal gradient difference over all pixels or detector elements can be used as characteristic-curve correction factor for the dark current. In general, instead of the mean value, any other suitable variable can also be used, for example a median or only one of the two signal gradients.
In a further advantageous configuration of the invention, the gain is determined from the difference of the two signal gradients. The difference in the two signal gradients precisely provides the photon-flux-dependent component of the signal, and so the latter can be used as gain.
Accordingly, the characteristic-curve correction factors for the gain can be formed from the mean difference of the signal gradients divided by the pixel-individual differences in the signal gradient. Thus, the gradient difference divided by the mean gradient difference over all pixels or detector elements can be used for each pixel. Instead of the mean difference, any other variable formed from the difference over all pixels can be used, for example the median or any other suitable variable.
In respect of the sequence of determining the individual characteristic-curve correction factors, it is advantageous if the characteristic-curve correction factors for the signal offset are determined first, followed by those for the dark current and lastly those for the gain. Determining the characteristic-curve correction factors for the dark current can contain the use of the characteristic-curve correction factors for the signal offset. Likewise, the characteristic-curve correction factors for the gain can be determined using the characteristic-curve correction factors for the dark current.
A more accurate correction of the signal offset can be achieved if the characteristic-curve correction factors of the signal offset are determined from the image values of both recording temperatures, for example by virtue of each being formed as a mean value from the two signal offsets. In reality, the two gradient lines will not cross exactly at the temporal origin of the integration. Therefore, the use of the mean value of both signal offsets will lead to a correction improvement over all pixels.
The two different temperatures of the emitter with a homogeneous temperature are expediently chosen in such a way that the temperature range there between at least predominantly covers the main detection spectral range of the matrix detector. Temperatures between 0° C. and 150° C., particularly between 30° C. and 120° C., are particularly expedient. The further temperatures or infrared colours of the matrix detector can be corrected in such a way that the characteristic-curve correction factors for such temperatures are interpolated from the two temperatures. Below, the interpolation also contains an extrapolation. The interpolation and extrapolation can be a dependence of the radiation flux on the temperature, either linearly or using a function.
The detector temperature of the matrix detector forms a further independent variable of the characteristic-curve correction factors. Thus, these are dependent on the detector temperature. In this respect, it is advantageous if the detector temperature of the matrix detector is determined and the characteristic-curve correction factors are determined as a function of the detector temperature. Expediently, the characteristic-curve correction factors are established for a plurality of different detector temperatures. Then, the characteristic-curve correction factors for other detector temperatures can be formed by interpolation or by extrapolation.
Moreover, the invention is directed to a method for controlling a guided missile, in which the characteristic-curve correction factors are established with the aforementioned method according to the invention, in particular. Advantageously, the characteristic-curve correction factors for the gain and, in particular, for the dark current as well are determined prior to the deployment of the guided missile, expediently by virtue of the matrix detector lying within a building. The characteristic-curve correction factors for the offset can be determined or corrected during a subsequent deployment.
The guided missile containing the matrix detector in its seeker head can be brought to a deployment site and can be launched there. Two images of the surroundings can be recorded with different integration times by the matrix detector prior to or after launch, and a signal offset can be established for each pixel from the two image values. On account of the greatly fluctuating scene, it is expedient if the two images with different integration times are recorded directly in succession. Now, a signal offset or the characteristic-curve correction factors for the signal offset can be established for each of the pixels from the two image values. The images can be corrected in a further step using the characteristic-curve correction factors for the signal offset and gain. The guided missile can be controlled using the images that were at least corrected in respect of the gain.
In order to obtain swift target tracking or control of the guided missile, it is advantageous if the two images with different integration times are already recorded while the matrix detector is cooled. By way of example, cooling is implemented to establish a readiness of the matrix detector. In this way, the signal offset or its characteristic lines can already be determined when the matrix detector is not yet fully operational.
Moreover, the invention is directed to a guided missile with a seeker head containing a matrix detector, a data memory and a control unit for controlling a deployment flight. In order to reliably correct images for use in the guided missile controller, the data memory, according to the invention, contains characteristic-curve correction factors for the gain of the detector elements of the matrix detector. Expediently, these were obtained according to the above-described method according to the invention.
Independently of the method used to establish the characteristic-curve correction factors for the gain, the dark current and/or the signal offset of the detector elements of the matrix detector, it is advantageous if the control unit is prepared to determine characteristic-curve correction factors for the signal offset of the detector elements of the matrix detector on the basis of two scene images with different integration times recorded directly in succession during a deployment. As a result of this, image artefacts on account of the signal offset fluctuating in the meantime can be kept low.
The description of advantageous configurations of the invention provided above contains numerous features, some of which are reproduced in combination as a plurality in a few dependent claims. However, expediently, the features may also be considered individually and combined to form expedient further combinations, in particular in the case of dependency references of claims, and so an individual feature of a dependent claim is combinable with a single feature, a number of features or all features of another dependent claim. Moreover, these features are combinable, respectively on their own and in any suitable combination, both with the method according to the invention and with the apparatus according to the invention in accordance with the independent claims. Thus, method features should also be considered to be phrased in substantive fashion as properties of the corresponding apparatus unit and functional apparatus features should also be considered to be phrased as corresponding method features.
The above-described properties, features and advantages of this invention and the manner in which they are achieved will become clearer and more easily understandable in conjunction with the following description of the exemplary embodiments, which are explained in more detail in conjunction with the drawings. The exemplary embodiments serve to explain the invention and do not restrict the invention to the combination of features specified therein, not even in relation to functional features. Moreover, features of each exemplary embodiment which are suitable therefore may also be explicitly considered in isolation, may be removed from an exemplary embodiment, introduced into another exemplary embodiment for the purposes of complementing the latter and/or be combined with any one of the claims.
Other features which are considered as characteristic for the invention are set forth in the appended claims.
Although the invention is illustrated and described herein as embodied in a method for determining characteristic-curve correction factors of a matrix detector that images in the infrared spectral range, it is nevertheless not intended to be limited to the details shown, since various modifications and structural changes may be made therein without departing from the spirit of the invention and within the scope and range of equivalents of the claims.
The construction and method of operation of the invention, however, together with additional objects and advantages thereof will be best understood from the following description of specific embodiments when read in connection with the accompanying drawings.
Referring now to the figures of the drawings in detail and first, particularly to
For production reasons, the signals of the individual detector elements of the matrix detector 8 are inhomogeneous in relation to one another in terms of the signal. Therefore, they supply a different signal when exactly the same photon flux strikes. This leads to image aberrations, which can make interpreting the object significantly more difficult, particularly if the latter still is far away from the guided missile 2. For the purposes of correcting such artefacts, characteristic-curve correction factors for each detector element of the matrix detector 8 can be stored in a data memory 16 that is accessible to the control unit 10. The signal of the individual detector elements now can be corrected therewith, and so the image aberrations are reduced.
Subsequently, two images of this area 20 are recorded, with the images being recorded with different integration times. Subsequently, a further area or the same area 20 with a different homogeneous temperature is imaged on the matrix detector 8 and two images with different integration times are recorded again.
A result of this method step as illustrated schematically in
The signal s is substantially composed from three signal components: the signal offset ω, the signal gain or simply gain γ, and the dark current θ. The signal offset ω is constant over the integration time t, as illustrated in
In the subsequent method step 24, the gradients σ1 and σ2 of the signals s are determined as a function of the integration time t. In
Now, the signal offset ω can be determined for each detector element and a reference value for the signal offset can be determined therefrom. Then, the characteristic-curve correction factors Ω for the signal offset for each detector element emerge from the differences of the individual signal offsets ω from the reference value. For the further calculations, the signal offset ω can be subtracted from the signal s or the measurement points 22. However, this is optional.
The gain γ for the detector element is determined in the next method step 28. The gain γ depends on the blackbody temperature TBB and emerges from the difference of the two gradients 6, as indicated in
The dark current θ for the individual pixels is determined in a further method step 30 and characteristic-curve correction factors Θ for the dark current can be determined therefrom with knowledge about all values for the dark current θ. In relation to the gradients σ, the dark current θ adopts the role of an offset. It can be subtracted as a gradient constant, and so the following arises: σ(1)=σ−θ. σ(1) is the first correction to the gradient σ in this case. Here, the absolute gradient of the dark current θ need not be known and can be set to a value, for example the gradient σ1 at the lower blackbody temperature TBB1, or the mean gradient value of the two gradients σ1 and σ2. This is shown in exemplary fashion in
Now, the three variables of the signal offset ω, dark current θ and gain γ are known for the corresponding detector element. This procedure can be adopted for a multiplicity of detector elements or for all detector elements. From this, it is possible to determine reference values for the signal offset, dark current and gain, from which the individual characteristic-curve correction factors Ω, Θ and Γ for the signal offset, dark current and gain for each detector element are able to be determined with the individual values.
In order now to arrive at a corrected image with a comparable brightness to the original image, the three variables of signal offset Ω, dark current Θ and gain Γ can be selected in such a way that the corrected signal for each variable is corrected to the mean value of the variable. By way of example, if Θ=σ(TBB1) is formulated, a corrected image according to
In respect of specific calculations, the characteristic-curve correction factors can be determined as set forth below. Initially, the offset ω and the gradients σ in relation to the integration time t are calculated from the four measurement values 22 s(t1/2, TBB1/2) at two different integration times t1 and t2 and two blackbody temperatures TBB1 and TBB2:
σ(TBB1)=(s(t2,TBB1)−s(t1,TBB1))/(t2−t1)
σ(TBB2)=(s(t2,TBB2)−s(t1,TBB2))/(t2−t1)
ω(TBB1)=s(t1,TBB1)−t1σ(TBB1)
ω(TBB2)=s(t1,TBB2)−t1σ(TBB2).
According to the model, ω(TBB1)=Ω(TBB2) should apply; this is usually satisfied to a good approximation. However, the slightly more improved mean value ω=(ω(TBB1)+ω(TBB2))/2 is used for the corrections. Naturally, this does not apply to the gradients σ; they depend on the blackbody temperature TBB. According to the model, the following applies thereto: σ(TBB)=γf(Φ(TBB))+θ.
Now, the individual gain and dark current coefficients γ and θ should be determined therefrom, the coefficients, moreover, being dependent on the detector temperature. By way of example, the matrix detector 8 was cooled to 104 K. Proceeding from the two equations:
σ(TBB1)=γf(Φ(TBB1))+θ
σ(TBB2)=γf(t(TBB2))+θ,
the difference thereof is calculated to this end:
Δ(TBB1,TBB2):=σ(TBB2)−σ(TBB1)=γ(f(Φ(TBB2)−f(Φ(TBB2)).
The dark current component drops out as a result of forming the difference. The multiplicative correction terms Γ in gain for the detector elements are then provided individually by
Γ=Mean(Δ(TBB1,TBB2))/Δ(TBB1,TBB2),
Where Mean( . . . ) denotes robust averaging over the entire image and ensures the normalization of the corrections to mean value of 1. The normalization could also be carried out in logarithmic fashion, which takes account of the multiplicative character of the Γ-correction to a slightly better extent.
Calculating the dark current component Θ remains to be done, the latter being independent of the scene temperature but subsequently being included in multiplicative fashion with the integration time into the calculation of the characteristic-curve correction factors. The exact extrapolation to f(Φ=0) or else f=0, which in turn would require knowledge of f, is once again not necessary for the correction since one could just as easily take account of a constant offset f(Φ) the dark current component as well (even if the latter is weighted by γ, it only needs to be constant). Therefore, even negative values of Φ are not problematic from a formal point of view. As a reference point for the correction,
θ=(σ(TBB1)+σ(TBB1))/2
is formulated as mean value of the two gradient images. It would also be possible to use each of the two gradient images; however, the aforementioned approach is symmetrical and supplies slightly better results.
The dark current deviations are corrected by Θ=Mean(θ)−θ, as a result of which the mean correction is zero.
The sets of coefficients (“images”) Ω, Γ, Θ are available as a result of the laboratory measurement. Here, the two integration times t and the two detector temperatures TBB chosen for the laboratory measurement are arbitrary in principle, but should be chosen in such a way that, firstly, they cover the typical dynamic range well and, secondly, do not lie in the non-linear range of the detector characteristic curves.
A further parameter is the detector temperature TD, which was not considered previously and was formulated as a constant. In order to be able to carry out a characteristic-curve correction at different detector temperatures, particularly during the cool down, measurements would be accordingly recorded at different detector temperatures TD and corresponding interpolation coefficients would be stored in the missile 2 such that
Ω=Ω(TD), Γ=Γ(TD), Θ=Θ(TD)
are present and are used according to the current detector temperature TD.
Since the offset image Ω may not be as stable in time as the gain image Γ, the integration-time-independent offsets need not be stored as they can be calculated dynamically from the double images s(t1), s(t2), in a manner analogous to the laboratory measurement:
Ω=s(t1)−t1(s(t2)−s(t1))/(t2−t1).
With all three parameter sets being present, the corrected image scorr when applying the characteristic-curve correction to the initial image s, recorded with an integration time t, emerges as
In this form, it is very clear that the offset correction is composed of the integration-time-independent Ω and the dark current integrated over time tΘ. Addition of the mean value images Mean(Ω)+t Mean(Θ) ensures that the mean value, i.e., Mean(s)=Mean(scorr), is obtained.
By way of example, the guided missile 2 or any other missile can be controlled on the basis of the characteristic-curve correction factors Ω, Γ, Θ. Such a method is illustrated in
In the subsequent method step 34, the guided missile 2 can be transported to a deployment site, or else it is initially transported to a storage site and, from there, to a deployment site at a later time. When the guided missile 2 is deployed 36, the matrix detector 8 is cooled to its deployment temperature, for example to less than 150 Kelvin.
The characteristic-curve correction factors Ω for the signal offset are established in method step 38 before, during or after the cool down of the matrix detector 8 to its deployment temperature. This is implemented using at least two images that were recorded immediately in succession and from which the gradient σ, as described above, is established. In this case, the two images are not of an area of homogeneous temperature but may originate from the surrounding scene around or in front of the guided missile 2. Since the signal offset is independent of the integration time and the temperature of the recorded scene, substantially any scene is suitable for calculating the characteristic-curve correction factors Ω for the signal offset. It is expedient to assign the latter to the detector temperature TD and optionally carry out the method for two different detector temperatures TD in order to be able to determine the signal offset even at the deployment temperature of the matrix detector 8 by extrapolation.
Subsequently, the guided missile 2 is launched and the object scene in front of the seeker head 4 of the guided missile 2 is imaged on the matrix detector 8. In method step 40, the signals s of the individual pixels of the matrix detector 8 are corrected on the basis of the characteristic-curve correction factors Ω, Γ, Θ and the image obtained from these corrected signals s is analysed by the control unit 10 in respect of controlling the guided missile 2 to a target emitting in the infrared range, for example. Here, the characteristic-curve correction factors Ω, Γ, Θ are available in the data memory 16, either as a function depending on the detector temperature TD or as a multi-dimensional dataset, by means of which the characteristic-curve correction factors Ω, Γ, Θ can be calculated by interpolation to the real detector temperature.
The following is a summary list of reference numerals and the corresponding structure used in the above description of the invention:
| Number | Date | Country | Kind |
|---|---|---|---|
| 102018001076.8 | Feb 2018 | DE | national |
