Patent Grant
8067735
Embodiments of the present invention pertain to infrared imaging systems and, more particularly, to such systems using an imager algorithm to limit shutter actuations of the systems without sacrificing image quality.
As is known, infrared cameras generally employ a lens working with a corresponding infrared focal plane array (FPA) to provide an image of a view in a particular axis. The operation of such cameras is generally as follows. Infrared energy is accepted via infrared optics, including the lens, and directed onto the FPA of microbolometer infrared detector elements. Each detector element responds to the heat energy received by changing its resistance value. An infrared (or thermal) image can be formed by measuring the detector elements' resistances—via applying a voltage to the detector elements and measuring the resulting currents or applying current to the detector elements and measuring the resulting voltages. A frame of image data may, for example, be generated by scanning all the rows and columns of the FPA. A dynamic thermal image (i.e., a video representation) can be generated by repeatedly scanning the FPA to form successive frames of data, with such frames being produced at a rate sufficient to generate a video representation of the thermal image data.
Individual detector elements have unique response characteristics. These response characteristics are found to produce non-uniformities, which often result in fixed pattern noise. Many infrared cameras have functionality to correct for such noise. For example, some infrared cameras can automatically or manually perform offset compensation, which corrects for variations in the individual detector element responses by observing a uniform thermal scene (e.g., by placing a shutter between the optics and the array) and measuring offset correction data for each detector element which provides the desired uniform output response. These measured offset corrections are stored, then later applied in subsequent infrared measurements (e.g., with the shutter open) to correct for fixed pattern noise. Other compensations can also be applied, such as 2-point correction.
As is known, offset compensation functionality is found in most conventional infrared cameras because it leads to improved imaging capabilities. However, offset compensation can be an inconvenience to the user as it necessitates activation of the camera shutter, thereby “freezing” the camera image for a short period of time when the shutter is closed. Therefore, it is desirable to keep the period between offset compensations lengthy so as to limit the general inconvenience to the user of the camera, while still maintaining good image quality.
Temperature changes within or surrounding an infrared camera can be further found to result in the individual detector elements exhibiting their unique response characteristics. In particular, the change in temperature of the camera's internal components, e.g., due to self-heating or as the result of changes to the surrounding ambient temperature, leads to the individual detector elements exhibiting fixed pattern noise over extended lengths of time. For example, during initial powering of an infrared camera, the internal components can be found to continue to rise in temperature for a period of time before the camera becomes thermally stable. Because of this, offset compensation is often performed at an increased frequency during such period so as to maintain good image quality from the camera. Such increased frequency of offset compensation correspondingly results in an increased frequency of shutter actuation. Consequently, there is further inconvenience for the user as the shutter is closed more often during such period.
What are needed are apparatus and systematic methods to address or overcome one or more of the limitations briefly described above with respect to offset compensation functionality in infrared imaging systems.
Embodiments of the invention involve an infrared imaging system having functionality for maintaining image quality in the presence of temperature drift of the system. The functionality is applied repetitively to maintain image quality of a target scene, yet without continuous actuation of a shutter of the system. The functionality of the imaging system results from implementing an imager algorithm. In use, the imager algorithm functions with a calibration curve created for the imaging system, with the curve comprising a plot of system output versus target scene temperature.
In certain embodiments, the imaging system includes a focal plane array (FPA); an enclosure disposed over the FPA, yet allowing for infrared energy from a target scene to be directed onto the FPA; and a shutter that can be selectively actuated so as to block infrared energy emanating from the target scene from reaching the FPA. When the imaging system is used in the field to view a target scene, the shutter of the system is initially closed to ascertain a base output for the detector elements of the FPA. Used in conjunction with the calibration curve and real-time measurements from the imaging system, the base output serves as a reference measurement in calculating the output of the FPA detector elements attributable to the target scene on an ongoing basis without necessitating further actuation of the shutter.
The following drawings are illustrative of particular embodiments of the present invention and therefore do not limit the scope of the invention. The drawings are not to scale (unless so stated) and are intended for use in conjunction with the explanations in the following detailed description. Embodiments of the present invention will hereinafter be described in conjunction with the appended drawings, wherein like numerals denote like elements.
The following detailed description is exemplary in nature and is not intended to limit the scope, applicability, or configuration of the invention in any way. Rather, the following description provides practical illustrations for implementing exemplary embodiments of the present invention. In the foregoing detailed description, the invention has been described with reference to specific embodiments. However, it may be appreciated that various modifications and changes can be made without departing from the scope of the invention as set forth in the appended claims.
As described above, infrared imaging systems are typically configured to perform offset compensation by actuating or closing a shutter thereof to reset offsets for detector elements of a focal plan array (FPA) based on the current temperature of the shutter. In turn, such offsets are used to correct non-uniformities from the individual detector elements of the FPA. However, one limitation to this compensation technique is the high frequency of shutter actuation that is warranted, particularly when ambient temperature surrounding the imaging systems varies or the internal components of the imaging systems are found to exhibit thermal change. As described above, high frequency of shutter actuation can inconvenience the user as the image provided by the infrared imaging system is often frozen for a short period of time when the shutter is closed. As alluded to above, embodiments of the invention are provided to eliminate the need to actuate a shutter during periods of continued use of the imaging system.
As is generally understood, the FPA 14 can include a plurality of infrared detector elements (not shown), e.g., including bolometers, photon detectors, or other suitable infrared detectors well known in the art, arranged in a grid pattern (e.g., an array of detector elements arranged in horizontal rows and vertical columns). The size of the array can be provided as desired. For example, an array of 160×120 detector elements can be employed, but the invention should not be limited to such. Further detailed description of an FPA and its functioning can be found in patent application Ser. No. 11/553,373, entitled “Multiple View Infrared Imaging System”, the disclosure of which is incorporated by reference herein in relevant part.
While not shown, one or more electrical circuits (not shown) are provided downstream from the FPA 14 to create an image based on the image view captured by the optics 12 and the corresponding infrared energy 16 projected onto the detector elements of the FPA 14. Further detailed description of such electrical circuits and their functioning can be found in patent application Ser. No. 12/054,818, entitled “Offset Compensation Scheduling Algorithm for Infrared Cameras,” the disclosure of which is incorporated by reference herein in relevant part.
Further included in the system 10, and generally disposed between the optics 12 and the FPA 14, are an enclosure 22 as well as a shutter 24. The enclosure 22, as shown in
The enclosure 22 defines opposing apertures, a first or outer aperture 26 and a second or inner aperture 28. Each of the apertures 26, 28 are in alignment with the FPA 14 such that the infrared energy 16 from the source or target scene 18 can be directed through the apertures 26, 28 and onto the FPA 14. In certain embodiments, the infrared imaging system 10 can be a camera, and as shown, the enclosure 22 can be operatively coupled to an outer surface 30 of such camera so that the inner aperture 28 is located near the FPA 14, while the outer aperture 26 is located away from the FPA 14. In certain embodiments, the enclosure 22 can be formed of one or more isothermal materials, e.g., aluminum, so that its temperature is uniformly provided across its surfaces; however, the invention should not be limited to such. As further described below, the enclosure 22 can just as well be formed of one or more non-isothermal materials.
In certain embodiments, as shown in
As should be appreciated, the detector elements of the FPA 14 are exposed to varied sources of infrared energy depending on whether the shutter 24 is in its closed position or its open position. In particular, when the shutter 24 is in its closed position, the detector elements of the FPA 14 are only exposed to infrared energy 34 emanating from the shutter 24 and the enclosure 22. Conversely, when the shutter 24 is in its open position, the detector elements of the FPA 14 are exposed to the infrared energy 16 emanating from the target scene 18 (via the outer and inner apertures 26, 28 of the enclosure 22), as well as the infrared energy 34 emanating from the enclosure 22 and the shutter 24. It should be appreciated that the thermal properties of the enclosure 22 and the shutter 24 are substantially similar in light of their close proximity and their similar compositions (e.g., both are generally formed of metal). When in its open position, the shutter 24 is found to lie adjacent to a corresponding surface portion of the enclosure 22. Accordingly, infrared energy 34 emanating from the enclosure 22 and the shutter 24 in its open position can be generally thought of as the energy emanating from the enclosure 22.
In general, by taking the output of the detector elements of the FPA 14 when the shutter 24 is in its closed position and subtracting such from the output of the detector elements of the FPA 14 when the shutter 24 is in its open position, one can selectively parse the output of the FPA 14 that is specifically associated with the target scene 18. As should be appreciated, such technique is well known and used in many conventional infrared cameras. However, as described above, such technique necessitates frequent actuating of the shutter or flag when camera temperature, e.g., either within or ambient thereto, fluctuates. To address this limitation, the infrared imaging system 10 of
As alluded to above, the curve 40 is generated by calculating output of the reference detector element (e.g., δ−Counts) on the y-axis for different temperatures of a target scene (e.g., ° C.) on the x-axis. As shown, in certain embodiments, output of the reference detector element can be quantified as magnitude of counts. Such counts, in certain embodiments, can be provided from an output of an analog-to-digital converter (ADC). As briefly alluded to above, the outputs of the FPA detector elements are transmitted through one or more electrical circuits of the system 10, with the circuits configured to generate an image based on infrared energy from the view passing through the optics 12 and there from directed onto the FPA 14. In certain embodiments, the signals from the detector elements would be amplified and converted from an analog form to a digital form in the electrical circuits (e.g., using an ADC) so the intensities of the signals can be quantified in terms of counts. Generally, the quantity of digital counts measured corresponds to thermal intensity borne on the detector elements of the FPA 14.
In one example, the target scene temperature for T1 (referenced as 42) may be 20° C. At this value of T1, a corresponding output of the reference detector element can be measured from the system 10. In certain embodiments, when the reference detector element represents a plurality of detector elements of the FPA 14, the measured outputs of the plurality of elements are averaged together to arrive at a single output for the reference detector element. In certain embodiments, the corresponding output may be derived from more than one measurement. For example, at such T1 value, measurements can be taken for the reference detector element both (i) when the shutter 24 is in its closed position and (ii) when the shutter 24 is in its open position. In turn, the corresponding reference detector element output, C1 (referenced as 44), is calculated by taking the measured output in counts from (i) above and subtracting it from the measured output from (ii) above. Accordingly, the output for the reference detector element attributable to the target scene (C1) is calculated relative to the target temperature at 20° C. (T1), using the following equation:
C1=δ−Counts(T1=20° C.)=Counts(shutter open)−Counts(shutter closed). (i)
Using the T1 and C1 values, a corresponding point 46 can be plotted in generating the calibration curve 40. The same process can then be repeated for other temperature values of the target scene in generating the curve 40. That is, for other target scene temperature values, e.g., T2 (referenced as 48) and T3 (referenced as 50), corresponding outputs for the reference detector element can be calculated, e.g., C2 (referenced as 52) and C3 (referenced as 54), from which corresponding points, 56 and 58, can be further plotted in generating the curve 40.
With further reference to the curve 40 of
During generation of the calibration curve 40 of
As described above, the temperatures of the enclosure 22 and the shutter 24 are substantially similar in light of their close proximity and their similar compositions. Accordingly, in certain embodiments, as shown in
Generating the calibration curve 40 of
As described above, the calibration curve 40 of
As described above, the temperature of the shutter 24 (TShutter, referenced as 60) is generally reflective of the temperature of the enclosure 22, which in certain embodiments, as described above, can be retrieved from the temperature sensor 36 located on the enclosure 22. With regard to this example, such shutter temperature (TShutter) provides a reference temperature on the x-axis of the calibration curve 40, which is reflective of the temperatures of the system structures proximate to the FPA 14. In turn, the imager algorithm of the system 10 uses the curve 40 in determining a corresponding reference output of the reference detector element (CtShutter, referenced as 62) for such shutter temperature (TShutter). As described above, such reference output of the reference detector element (CtShutter) represents an offset in output of the reference detector element in light of the variance in temperature proximate to the FPA 14 from its generally maintained temperature during generation of the curve 40. In turn, as further described below, the reference output of the reference detector element (CtShutter) is the point on the y-axis from which output of the reference detector element attributable to the target scene is projected. Accordingly, such reference output represents a reference offset for the system 10.
For example, the imager algorithm derives the output of the reference detector element attributable to the target scene 18 relative to its reference output (CtShutter). With reference to
Once the difference in output of the reference detector element (δcRef) is determined, the output of the reference detector element attributable to the target scene (CtTarget, referenced as 66) can be obtained by adding such output of the reference detector element (δcRef) to its reference output (CtShutter) and locating such combined output value on the y-axis. Subsequently, in using the calibration curve 40 with respect to the output of the reference detector element attributable to the target scene (CtTarget), a corresponding target scene temperature (TTarget, referenced as 68) can be located on the x-axis.
The following equation represents the output relationship described above, with reference to
CtTarget=CtShutter+δcRef. (ii)
From the above equation, it should be understood that CtShutter is used to compensate or offset the output of the reference detector element for a real-time temperature of the enclosure 22/shutter 24 as opposed to the temperature of such elements maintained during generation of the calibration curve 40. Accordingly, a reference or offset point is provided on the y-axis of the curve 40, from which output of the reference detector element attributable to the target scene (δcRef) is projected, with such output being derived relative to output measured during the most recent closure of the shutter 24.
As described above, while
With reference to
Following measurement of the outputs of the detector elements when the shutter 24 is closed, the shutter 24 is opened. As should be appreciated, the system 10, or rather, the optics 12 of the system 10, would be directed at the target scene 18 soon after the shutter 24 is opened. In turn, a distinct output is measured for each detector element of the FPA 14. Following such measurements, the imager algorithm of the system 10 derives portions of the just-measured detector element outputs attributable to the target scene 18. Such derivation, similar to what was exemplified for the reference detector element with respect to
As alluded to above, the detector elements of the FPA 14 have unique response characteristics. Put another way, each detector element of the FPA 14 has a unique gain component. The gain component for the reference detector element is generally considered to have a fundamental value of 1, while the gain components of the other detector elements of the FPA 14 may vary from such fundamental value. Gain components for each of the detector elements, collectively shown as Gi,j (referenced as 72), can be calculated by the manufacturer for the imaging system 10 and stored in memory (e.g., of a processor) of the system 10 during the system's calibration. The calculation of such gain components (Gi,j) would be known to those skilled in the art. In turn, the imager algorithm retrieves these gain components (Gi,j) from memory and respectively multiplies them by the derived output portions of the detector elements (δci,j), resulting in products collectively shown as δci,j·Gi,j (referenced as 74).
Similar to that described above with reference to
With reference to
The following equation represents the output relationship described above, with reference to
CtTarget=CtShutter+(δci,j·Gi,j). (iii)
From the above equation, it should be understood that CtShutter is used to compensate or offset each of the output portions of the detector elements for real-time temperatures of the enclosure 22/shutter 24. Accordingly, a reference or offset point is provided on the curve 40 from which outputs of the detector elements attributable to the target scene are projected (resulting in a distinct CtTarget value for each detector element). As described above, such output portions are derived relative to output measured immediately after a last (or most recent) activation of the shutter 24. In turn, the temperatures of the target scene 18 (each being a distinct TTarget value, referenced as 78) with respect to each of the projected outputs (each CtTarget value) can be found on the x-axis by the imager algorithm of the system 10 via use of the curve 40.
It should be appreciated that the shutter 24 may be manually closed for certain events. For example, the infrared imaging system 10 may be moved to another area so as to view another target scene and/or the system 10 may be turned off. Accordingly, following such events, in certain embodiments, the imager algorithm is configured to keep the shutter 24 initially closed upon further use of the system 10. As such, when the shutter is initially closed, new outputs of the detector elements are measured and stored, thereby replacing the prior stored outputs. In turn, such newly measured and stored outputs are used by the imager algorithm in maintaining the image quality of the system 10 during further use of the imaging system 10.
As alluded to above, the embodied infrared imaging system 10 is configured to maintain image quality in presence of temperature drift for the system 10 without frequent actuation of the shutter 24. With reference to
Accounting for such temperature variance, and its impact on the outputs of the FPA detector elements, is buoyed by the inclusion of the enclosure 22 within the imaging system 10. In particular, the temperature variance proximate to the FPA 14 is generally confined within the enclosure 22. Such confinement enables accurate accounting, and subsequent offsetting, of a drift component (attributable from the temperature variance) from the calculated outputs of the FPA detector elements over time. To that end,
As described above, the detector elements of the FPA 14 are exposed to varied sources of infrared energy depending on whether the shutter 24 is in its closed position or its open position. In particular, when the shutter 24 is in its closed position, the detector elements of the FPA 14 are only exposed to infrared energy 34 emanating from the shutter 24 and the enclosure 22. Conversely, when the shutter 24 is in its open position, the detector elements of the FPA 14 are exposed to the infrared energy 16 emanating from the source 18 (via the outer and inner apertures 26, 28 of the enclosure 22), as well as the infrared energy 34 emanating from the enclosure 22 and the shutter 24.
Because the detector elements of the FPA 14 have unique response characteristics, each of the elements are uniquely affected by the infrared energy 34 emanating from the shutter 24 and the enclosure 22. Accordingly, each detector element of the FPA 14 has its own distinct drift component, collectively represented as Δi,j (and referenced as 80). As described above with respect to
As can be gathered from
The imager algorithm, in turn, measures the outputs of the detector elements (δci,j, again referenced as 70); retrieves the gain components for each of the detector elements (Gi,j, again referenced as 72) from memory and respectively multiplies the detector element outputs by the gain components, leading to (δci,j·Gi,j) values (again referenced as 74); and adds these gain-corrected outputs (δci,j·Gi,j) to the output offset for the FPA detector elements (CtShutter), with these sums representing the corrected output for the detector elements (referenced as 82).
However, as described above, such corrected outputs 82 contain components not only of the scene 18, but also from any increase in infrared energy 34 (depicted in
CtTarget=CtShutter+(δci,j·Gi,j)−Δi,j. (iv)
It should be appreciated that during an initial image maintenance iteration by the imager algorithm (immediately after the most recent closure of the shutter 24), as depicted in
The drift components (Δi,j) are largely based on a difference in measurements taken by the imager algorithm between when the shutter 24 was last closed and a current state with the shutter 24 open. In certain embodiments, the derivation of the drift components (Δi,j) involves use of the temperatures of the enclosure 22 when the shutter 24 was last closed (TShutter (last taken (with shutter closed)) and in the present when the shutter 24 is open (TShutter (present (with shutter open))). In turn, the respective outputs corresponding to these enclosure temperatures (each obtained by using the curve 40) are used in calculating the drift components (Δi,j). In certain embodiments, the calculation of the drift components (Δi,j) further involves use of a drift coefficient, DShutter i,j, with the following equation being used:
Δi,j=DShutter i,j·[CtShutter (present (with shutter open))−CtShutter (last taken (with shutter closed))], (v)
where DShutter i,j is an array of calibration constraints (one for detector element) which correct for drift.
The drift coefficients (DShutter i,j) are generally calculated at time of calibration and stored in memory of the imaging system 10. Calculating for the drift coefficients (DShutter i,j) involves taking measurements from the imaging system 10 at a time “0”, immediately after the shutter 24 is opened (i.e., so there is no drift component (Δi,j)) and a later time (i.e., time “1”) with the shutter 24 still open but with variance to the temperature of the enclosure 22/shutter 24 from time “0” (i.e., so there is now a drift component (Δi,j)). As should be appreciated, such variance to the enclosure/shutter temperature impacts the outputs from the FPA detector elements taken at time “1”. As should be appreciated, such points in time (“0” and “1” times) have already been described above with respect to
CtShutter
In turn, using equation (v) above, the drift component (Δi,j) can be replaced in equation (vi), resulting in:
CtShutter
which can be reconfigured to solve for the drift coefficients (DShutter i,j), resulting in the following equation:
DShutter i,j=[Gi,j·[(δci,j)1−(δci,j)0]]/[CtShutter
With the drift coefficients (DShutter i,j) previously calculated and stored in memory of the imager system 10, the imager algorithm can calculate the drift components (Δi,j) using equation (v). Such drift component can be used by the imager algorithm in deriving output portions for each of the FPA detector elements attributable to the target scene (CtTarget values), using equation (iv) above. In turn, the imager algorithm uses these outputs (CtTarget values) with the curve 40 in determining the corresponding target scene temperatures (TTarget values). It should be appreciated that the above model provides good image quality for the imaging system 10 without the need to flag, or close the shutter 24, even in the presence of significant swings in temperature with respect to the enclosure 22/shutter 24.
It will be appreciated the embodiments of the present invention can take many forms, and it is not intended that the embodiments of the invention presented herein should limit the scope thereof.
This application is a continuation application of U.S. application Ser. No. 12/339,434, filed Dec. 19, 2008 and issued as U.S. Pat. No. 7,683,321, the teachings of which are incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6144031 | Herring et al. | Nov 2000 | A |
| 6515285 | Marshall et al. | Feb 2003 | B1 |
| 6730909 | Butler | May 2004 | B2 |
| 6879923 | Butler | Apr 2005 | B2 |
| 7030378 | Allen et al. | Apr 2006 | B2 |
| 7122788 | Owen et al. | Oct 2006 | B1 |
| 20010045516 | Emanuel et al. | Nov 2001 | A1 |
| 20020074499 | Butler | Jun 2002 | A1 |
| 20020162963 | Lannestedt et al. | Nov 2002 | A1 |
| 20050029453 | Allen et al. | Feb 2005 | A1 |
| 20060279632 | Anderson | Dec 2006 | A1 |
| 20080099683 | Schmidt et al. | May 2008 | A1 |
| 20080210872 | Grimberg | Sep 2008 | A1 |
| 20080302956 | Anderson | Dec 2008 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20100237245 A1 | Sep 2010 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12339434 | Dec 2008 | US |
| Child | 12711649 | US |
