Reference is made to commonly assigned copending application Ser. No. 11/021,258 entitled “Programmable Spectral Imaging System” by Marek W. Kowarz, James G. Phalen, and J. Daniel Newman, filed Dec. 21, 2004 and Ser. No. 11/316,857 entitled “Imaging System with a Programmable Spectral Switch” by Marek W. Kowarz, James G. Phalen, and J. Daniel Newman, filed Dec. 23, 2005, which are both incorporated fully herein by reference.
The present invention relates to imaging systems and, more particularly, to methods and apparatus for detecting and analyzing spectral polarimetric images.
Imaging systems are used to collect images. The collected images are often analyzed, e.g., to identify objects of interest within the images. Spectral imaging systems or polarimetric imaging systems may be employed to collect images in one or more spectral bands or in one or more polarimetric states (e.g., 0°, 45°, 90°, and 135°), respectively, to facilitate the identification of some objects during analysis.
Spectral imaging systems and polarimetric imaging systems are typically separate physical instruments that collect distinct images. In order to interpret an image collected by both a spectral imaging system and a polarimetric imaging system, images obtained by each of these systems need to be co-registered. Co-registering images from separate imaging systems, however, is difficult. This problem is further exacerbated by the ever increasing resolution of imaging systems.
The present invention is embodied in apparatus and methods for detecting and analyzing spectral polarimetric images. Spectral polarimetric images are detected in accordance with one aspect of the present invention by selecting spectral bands of light within light received from a light source and capturing polarimetric images of the selected spectral bands of light. Spectral polarimetric images are analyzed by converting spectral polarimetric images to Stokes images and selecting two or more of the Stokes images for analysis.
An apparatus for detecting light received from a light source in accordance with one aspect of the present invention includes a tunable spectral selector and a polarimetric imager optically coupled to the tunable spectral selector. The tunable spectral selector selects one or more spectral bands of light within the received light and the polarimetric imager receives and captures the selected one or more spectral bands of light.
Spectral polarimetric images representing the intensity of an image in each of two or more polarization states in each of two or more wavelength bands are analyzed in accordance with one aspect of the present invention by converting the spectral polarimetric images for each of the two or more polarization states in each of the two or more wavelength bands to Stokes images and selecting two or more Stokes images for analysis.
The invention is best understood from the following detailed description when read in connection with the accompanying drawings, with like elements having the same reference numerals. When a plurality of similar elements are present, a single reference numeral may be assigned to the plurality of similar elements with a small letter designation referring to specific elements. When referring to the elements collectively or to a non-specific one or more of the elements, the small letter designation may be dropped. This emphasizes that according to common practice, the various features of the drawings are not drawn to scale. On the contrary, the dimensions of the various features are arbitrarily expanded or reduced for clarity. Included in the drawings are the following figures:
In an exemplary embodiment, light 106 carries an image that is received by detector 100 through input slit 110. Light 106 received by detector 100 is directed to tunable spectral selector 102 where one or more spectral bands of light 106 are selected for detection. In an exemplary embodiment, input lens assembly 112 directs light 106 along optical path 114 through light path selector 116 (described in further detail below) positioned along optical path 114 toward tunable spectral selector 102.
Tunable spectral selector 102 illustrated in
In an exemplary embodiment, grating 118 is a transmission grating that disperses light 106 as it passes through the grating 118. In an alternative exemplary embodiment, grating 118 may be a reflection grating that disperses light 106 as it is reflected by grating 118.
In an exemplary embodiment, spectral selection device 122 is a reflective device that selectively diffracts/reflects light incident on imaging surface 122a to select the one or more spectral bands. In accordance with this embodiment, imaging surface 122a may include a plurality of addressable segments that selectively diffract/reflect dispersed light 120 responsive to a spectral selection signal (e.g., generated by processor 128, which is described below). In an exemplary embodiment, grating 118 and spectral selection device 122 are selected and positioned relative to one another such that grating 118 essentially “smears” the light 106 from input slit 110 across imaging surface 122a of spectral selection device 122. Thus, the wavelengths of light 106 are spread across imaging surface 122a for selection thereof.
An exemplary reflective device for use as spectral selection device 122 is an electromechanical conformal grating device consisting of ribbon elements suspended above a substrate by a periodic sequence of intermediate supports. The electromechanical conformal grating device is operated by electrostatic actuation, which causes the ribbon elements to conform around the support substructure, thereby producing a grating. This device is commonly referred to as the conformal GEMS device, or more simply as the GEMS device, with GEMS standing for Grating Electro-Mechanical System. The GEMS device provides high-speed digital light modulation with high contrast, high efficiency, and a relatively large addressable active region. The GEMS device can be fabricated as a linear device with a thin active area in order to diffract a thin line of the dispersed light (e.g., to select narrow wavelength bands) or can be fabricated with a relatively wide active area in order to diffract a wider segment of the dispersed light (e.g., to select wide wavelength bands).
The behavior of GEMS devices and systems is described in commonly assigned U.S. Pat. No. 6,307,663; in commonly assigned U.S. Pat. No. 6,411,425 entitled “Electromechanical Grating Display System With Spatially Separated Light Beams”; and in commonly assigned U.S. Pat. No. 6,678,085 entitled “High-Contrast Display System With Scanned Conformal Grating Device” all to Kowarz et al., all of which are incorporated by reference herein.
In an exemplary embodiment, each of cavities 208 (or a row of cavities) are individually addressable. When an addressable cavity 208 is off, the ribbon 202 above the cavity reflects incident beam 210 of light back along the path of incident beam 210 as reflected light 212 such as illustrated in
Alternative exemplary spectral selection devices 116 include a Digital Micromirror Device (DMD) used in Digital Light Processor (DLP) systems manufactured by Texas Instruments, Inc., Dallas, Tex. and known Liquid Crystal on Silicon (LCOS) light modulators. Other suitable spectral selection devices will be understood by one of skill in the art from the description herein and are considered within the scope of the present invention.
Referring back to
Referring back to
In an exemplary embodiment, first beam splitter 402a is a 50/50 beam splitter that splits the incident light substantially equally, second beam splitter 402b is a 45°/135° polarimetric beam splitter, and third beam splitter 402c is a 0°/90° polarimetric beam splitter. The beam splitters and sensors are arranged such that first sensor 404a captures the 0° polarimetric angle of the spectrally selected light, second sensor 404b captures the 90° polarimetric angle of the spectrally selected light, third sensor 404c captures the 45° polarimetric angle of the spectrally selected light, and fourth sensor captures the 135° polarimetric angle of the spectrally selected light. Although the 0°, 45°, 90°, and 135° polarimetric angles are captured, as will be recognized by one of skill in the art from the description herein, essentially any angles and number of angles of light may be sensed by repositioning the beam splitters and sensors and/or using different beam splitters.
In an alternative exemplary embodiment, the polarimetric imager 104 (
Referring back to
At block 506, one or more spectral bands of light within the received light is selected responsive to a spectral selection signal. In an exemplary embodiment, processor 128 generates the spectral selection signal and spectral selection device 122 selects the one or more spectral bands of light responsive to the spectral selection signal. The spectral selection may involve dispersing the received light, selectively diffracting the dispersed light to select the one or more bands of light, and collimating the selectively diffracted one or more bands of light for capturing of the at least one polarimetric image. Grating 118 may disperse the bands of light for selection by spectral selection device 122. In addition, the grating 118 may collimate the light as diffracted/reflected by spectral selection device 122.
At block 508, the spectrally selected light is directed to polarimetric imager 104. In an exemplary embodiment, light path selector 116 directs the spectrally selected light to polarimetric imager 104, e.g., via spectrally selected lens assembly 134, which focuses the spectrally selected light onto a surface of polarimetric imager 104. In an exemplary embodiment, light path selector 116 passes light received through input slit 110 to spectral selection device 122, passes light reflected by spectral selection device 122 away from polarimetric imager 104, and reflects light diffracted by spectral selection device 116 toward polarimetric imager 104 to direct the spectrally selected light to polarimetric imager 104.
At block 510, polarimetric imager 104 captures at least one polarimetric image of the selected one or more spectral bands of light. Thus, spectral polarimetric detector 100 detects spectral polarimetric images. In an exemplary embodiment, polarimetric imager 104 at least substantially simultaneously captures polarimetric images having two or more polarimetric states (e.g., four polarimetric angles such as 0°, 45°, 90°, and 135°).
At block 512, the spectral polarimetric images are analyzed. The spectral polarimetric images may be analyzed by processor 128, e.g., by comparison to other stored images of known objects of interest, or may presented on display 132 for analysis by a human operator.
At block 602, a calibration matrix is generated for each of the two or more wavelength bands. In an exemplary embodiment, each calibration matrix is a Mueller matrix generated using spectral polarimetric image data obtained from polarized calibration images using the spectral polarimetric detector 100 of
As will be understood by one of skill in the art from the description is herein, an output Stokes vector (e.g., [S0-S3]out) is related to an input Stokes vector (e.g., [S0-S3]in) through a Mueller matrix (e.g., [M11-M44]) as illustrated in equation 1 below. Thus, the value of the Mueller matrix may be determined by measuring the output Stokes vector when the Stokes vector of the incoming image is known. The number of measurements depends on the number of Stokes vectors. For example, 16 Stokes measurements are used for a set of four Stokes vectors S0 . . . S3 and nine Stokes measurements are used for a set of three Stokes vectors. Depending on the configuration of the polarimetric filters, the measurements can be made sequentially or in a parallel fashion. Specifically, for a spectral polarimetric detector capable of capturing all four vectors simultaneously, four separate measurements may be made in a parallel fashion.
In a spectral polarimetric detector, it is desirable to measure the Mueller matrix for each distinct wavelength band of the system. Thus, the measurement and calculation of the Mueller matrix are desirably repeated over each wavelength range used. This process is described in further detail below with reference to
The Mueller matrix is used to calculate the Stokes images from a collected spectral polarimetric images. The Mueller matrix is essentially a calibration matrix for the polarization effects of the imager. In an ideal system, the matrix would has values of 1 along the diagonal axis as follows:
This indicates that each of the polarizers is perfect and does not let any light pass which does not match the polarization state of the polarizer. In practice this may not be the case and there may be some values that are not along the diagonal axis that are non-zero. This is partially due to the filters not being perfect and the optics before the polarizers changing the polarization state of the calibrated light sources (e.g., most optics will slightly depolarize light passing thru them). The advantage of using the Mueller matrix to calculate the Stokes images is that the Mueller matrix corrects for imperfections within the system and provides a calibrated Stokes image. If it is determined that the spectral polarimetric system is near ideal, the step of block 602 may be omitted.
At block 604, spectral polarimetric images are obtained. In an exemplary embodiment, the spectral polarimetric images are intensity images of each polarization state, e.g., I0, I1, I2, I3, in each spectrally selected wavelength band, e.g., λ0-λ6, obtained using the spectral polarimetric detector 100.
At block 606, the spectral polarimetric images are converted to Stokes images. The intensity images of each polarization state of the spectral polarimetric images may be converted to Stokes images using equations 3-6.
S0=2I0 (3)
S1=2I1−2I0 (4)
S2=2I2−2I0 (5)
S3=2I3−2I0 (6)
At block 608, the Stokes images are calibrated using the calibration matrices generated at block 602. If the spectral polarimetric detector 106 is near ideal, the calibration of step 608 may be omitted. Additional information describing the conversion of spectral polarimetric images to Stokes images of step 606 and the calibration of the Stokes images in step 608 are described in further detail below with reference to
At block 610, the Stokes images are saved, e.g., in memory 130 by processor 128. Is will be recognized by one skilled in the art that the Stokes images may be saved before calibration with the Stokes images being calibrated at a later time. Thus, the step of block 610 may be performed prior to the step of block 608.
At block 612, two or more of the Stokes images are selected for display. In an exemplary embodiment, three of the Stokes images are selected for presentation on a display 132. The use of three Stokes images may be selected since, in general, humans are used to seeing only three combinations of colors at a time (Red, Green, Blue) on a monitor. Thus, three bands may be selected from the multiple bands to display (e.g., as Red, Green, Blue combinations on a display 132 of a monitor). For example, for four (4) Stokes images for each of the 26 wavelength bands (e.g., AS0, AS1, AS2, AS3, BS0, BS1, BS2, BS3, . . . ZS0, ZS1, ZS2, ZS3), any of these bands may be selected for display in the Red, Green and Blue channels of a monitor (DS0=Red, KS0=Blue and QS0=Green or DS0=Red, DS1=Blue and DS2=Green, etc.). The selection of the exact band and polarization combinations can be performed by an algorithm that finds anomalies or by a priori knowledge of the object/target/signature of interest.
Although the invention is illustrated and described herein with reference to specific embodiments, the invention is not intended to be limited to the details shown. Rather, various modifications may be made in the details within the scope and range of equivalents of the claims and without departing from the invention.
Number | Name | Date | Kind |
---|---|---|---|
4776692 | Kalawsky | Oct 1988 | A |
4912059 | Newman et al. | Mar 1990 | A |
5005977 | Tomoff | Apr 1991 | A |
5011295 | Krishnan et al. | Apr 1991 | A |
5046850 | Tomoff | Sep 1991 | A |
5131742 | Schaff | Jul 1992 | A |
5247176 | Goldstein | Sep 1993 | A |
5296913 | Heffner | Mar 1994 | A |
5317150 | Norris et al. | May 1994 | A |
5440390 | Tirri | Aug 1995 | A |
5519493 | Reiley | May 1996 | A |
5666201 | Johs et al. | Sep 1997 | A |
5706212 | Thompson et al. | Jan 1998 | A |
5737298 | Suhan | Apr 1998 | A |
5956145 | Green et al. | Sep 1999 | A |
5963327 | He et al. | Oct 1999 | A |
6046805 | Kawamura et al. | Apr 2000 | A |
6166807 | Kawamura et al. | Dec 2000 | A |
6175412 | Drevillon et al. | Jan 2001 | B1 |
6177995 | Compain et al. | Jan 2001 | B1 |
6181421 | Aspnes et al. | Jan 2001 | B1 |
6211957 | Erdogan et al. | Apr 2001 | B1 |
6268916 | Lee et al. | Jul 2001 | B1 |
6327037 | Chou et al. | Dec 2001 | B1 |
6370407 | Kroeger et al. | Apr 2002 | B1 |
6466320 | Kawamura et al. | Oct 2002 | B1 |
6535286 | Green et al. | Mar 2003 | B1 |
6549282 | Johs et al. | Apr 2003 | B1 |
6583410 | Seddon | Jun 2003 | B1 |
6608587 | Sparrow et al. | Aug 2003 | B1 |
6640116 | Diab | Oct 2003 | B2 |
6704106 | Anderson et al. | Mar 2004 | B2 |
6713753 | Rovira et al. | Mar 2004 | B1 |
6717665 | Wagner et al. | Apr 2004 | B2 |
6734968 | Wang et al. | May 2004 | B1 |
6744509 | Davis et al. | Jun 2004 | B2 |
6762713 | McMillan et al. | Jul 2004 | B1 |
6765672 | Veasey | Jul 2004 | B1 |
6768971 | Sparrow et al. | Jul 2004 | B1 |
6795184 | Herzinger et al. | Sep 2004 | B1 |
6798514 | Daniels | Sep 2004 | B2 |
6801320 | Szafraniec | Oct 2004 | B2 |
6804003 | Wang et al. | Oct 2004 | B1 |
6804004 | Johs et al. | Oct 2004 | B1 |
6816260 | Peupelmann et al. | Nov 2004 | B2 |
6836327 | Yao | Dec 2004 | B1 |
6856398 | Ruchet | Feb 2005 | B2 |
6867853 | Ro et al. | Mar 2005 | B2 |
6885333 | Sparrow et al. | Apr 2005 | B2 |
6917427 | Krause et al. | Jul 2005 | B2 |
6927888 | Garcia et al. | Aug 2005 | B2 |
6950611 | Erdogan et al. | Sep 2005 | B2 |
7420675 | Giakos | Sep 2008 | B2 |
20020186468 | Lee et al. | Dec 2002 | A1 |
20030117624 | Daniels | Jun 2003 | A1 |
20030174328 | Russell et al. | Sep 2003 | A1 |
20030223064 | Anderson et al. | Dec 2003 | A1 |
20040012853 | Garcia et al. | Jan 2004 | A1 |
20040036876 | Davis et al. | Feb 2004 | A1 |
20040064064 | Zhou et al. | Apr 2004 | A1 |
20040070766 | Szafraniec | Apr 2004 | A1 |
20040145728 | Kim et al. | Jul 2004 | A1 |
20050062966 | Chen et al. | Mar 2005 | A1 |
20050105088 | Garcia-Caurel et al. | May 2005 | A1 |
20050264813 | Giakos | Dec 2005 | A1 |