The currently described invention relates to methods and apparatus for increasing the number of pixels delivered by a focal plane detector array.
Conventional electronic imaging techniques rely on the number of detectors that can be assembled together in a single array. Current manufacturing techniques limit detector resolution and how tightly the detectors can be spaced in an array. In addition, applications for detector arrays continue to call for detector arrays having smaller physical footprints while increasing pixel density. A need therefore exists for improved methods and systems for increasing pixel density delivered by detector arrays.
One embodiment is a method for increasing the pixel count delivered from a plurality of light detectors when scanning an object. The method includes positioning a mask, having a plurality of two dimensional Walsh-Hadamard filter patterns, in front of a plurality of light detectors to control the passage of light through the mask filter patterns to the light detectors. The method also includes positioning a lens assembly in front of the mask to project an image through the mask filter patterns on to the plurality of light detectors. The method also includes moving the mask, plurality of light detectors, or the image in a plane defined by a planar surface of the plurality of light detectors. The method also includes measuring the filtered image projected through the mask with the plurality of light detectors.
In some embodiments, the method includes storing a set of data representing the measured filtered image. In some embodiments, the method includes retrieving a representation of the projected image from the stored set of data by performing an inverse Walsh-Hadamard transform on the stored set of data. In some embodiments, the method includes moving the mask relative to the plurality of light detectors and the image.
In some embodiments, the method includes moving the plurality of light detectors relative to the mask and image. In some embodiments, the method includes moving the image relative to the plurality of light detectors and the mask. In some embodiments, the method includes moving one of the mask, plurality of light detectors, or the image along one or more axes.
Another embodiment is an apparatus for scanning an object. The apparatus includes a mask that includes a plurality of two dimensional Walsh-Hadamard filter patterns to control the passage of light through the mask. The apparatus also includes a plurality of light detectors located on a first side of the mask to measure light passing through the mask. The apparatus also includes a lens assembly located on a second side of the mask to project an image through the mask on to the plurality of detectors to generate a filtered image. The apparatus also includes an actuator to move the mask, plurality of light detectors, or the image.
In some embodiments, the actuator moves in a step and repeat motion. In some embodiments, the mask is coupled to the plurality of light detectors. In some embodiments, the actuator is configured to move one of the mask, plurality of light detectors, or the image along one or more axes.
In some embodiments, the actuator is configured to move the mask relative to the plurality of light detectors and the image. In some embodiments, the actuator is configured to move the plurality of light detectors relative to the mask and image. In some embodiments, the actuator is configured to move the image relative to the plurality of light detectors and the mask. In some embodiments, the apparatus includes a computer memory configured to store a set of data representing the measured filtered image. In some embodiments, the apparatus includes an image displacing element to move the image relative to the plurality of light detectors and the mask.
Other aspects and advantages of the current invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating the principles of the invention by way of example only.
The foregoing features of various embodiments of the invention will be more readily understood by reference to the following detailed descriptions in the accompanying drawings, in which:
Embodiments disclosed herein significantly increase the number of pixels delivered by a standard focal plane detector array. The increase is the result of applying Walsh-Hadamard masks, in the form of micro masks, to the focal plane imagery. By using the Walsh-Hadamard masks it is possible to increase the number of pixels delivered by each individual detector by a factor corresponding to the pattern mask subarray size. For example, a pattern mask having a 4×4 Walsh-Hadamard array size increases pixel density by a factor of 16 (4×4=16).
The detector array 132 includes a plurality of light detectors 136 for measuring the light passing through the mask 104 from the second side 124 of the mask 104 to the first side 128 of the mask. In this embodiment, the mask 104 is actuated to move along a single axis 112 parallel to the face 148 of the light detectors 136. The apparatus 100 also includes a processor 150 coupled to the actuator 108 and the detector array 132. The processor 150 provides commands to the actuator 108 to controllably move the mask 104. The processor 150 also receives the outputs from the light detectors 136 of the detector array 132. The processor 150 includes computer memory and stores the light detector 136 outputs in the computer memory for subsequent processing and/or retrieval.
The apparatus 100 also includes a lens assembly 120 located on the second side 124 of the mask 104. The lens assembly 104 projects an image 144 of an object 140 through the mask 104 on to the plurality of light detectors 136 to generate a filtered image.
Referring to
Walsh-Hadamard (WH) matrices provide a set of orthogonal basis vectors which can be used to fully represent sampled signals which have 2n samples. In one embodiment, the WH matrices are developed (e.g., performing a WH transform) by starting with a primitive 2×2 matrix of the following form:
This primitive matrix is then expanded by replacing each of the terms in the primitive matrix with the primitive matrix, itself. The result of the first expansion is a type of outer product matrix of the following form:
Each of the rows in EQNS. 1 and 2 is an orthogonal WH basis vector. Thus, from EQN. 1 we have the four vectors:
h
1=1,1,1,1 EQN. 3
h
2=1,−1,1,−1 EQN. 4
h
3=1,1,−1,−1 EQN. 5
h
4=1,−1,−1,1 EQN. 6
This process can be continued to produce eight basis vectors of eight elements, sixteen basis vectors of sixteen elements, and so on.
In some embodiments, a specific type of WH basis function is used (Binary Walsh (BW) vectors) for non-coherent optical applications where it is difficult to create a mask with −1 openings. In a BW matrix (and function set), the −1 values are replaced by 0. Thus, EQNS. 3, 4, 5 and 6 convert to;
w
1=1,1,1,1 EQN. 7
w
2=1,0,1,0 EQN. 8
w
3=1,1,0,0 EQN. 9
w
4=1,0,0,1 EQN. 10
Each of these basis vectors can be arranged in any desired geometrical pattern. The basis vectors of EQNS. 7, 8, 9 and 10 could be arranged in 2×2 patterns. In this embodiment, we focus on sixteen element basis vectors for the mask 104. The principles discussed here are applicable to any BW and WH basis function set of any 2n size.
Referring to
With each step and dwell of the mask 104, the pattern of light falling on a given detector 136 is spatially modulated by the pattern block 116 immediately above the detector 136. The detector 136 thus reports out the corresponding Walsh-Hadamard filter pattern sequency component. The sequency of a Walsh-Hadamard function is a property analogous to the frequency of a sinusoid. A sequency ordered Walsh-Hadamard matrix is arranged so the number of sign changes in a row is in increasing order. For example, referring to
The following code fragments in C++ show the process of encoding and decoding a sampled function f[i] (i.e., creating a BW basis vector set, followed by expansion of f[i] in terms of these bases (i.e., applying a BW transform), to make a transformed function F[j]. Then, the process is inverted using a WH basis vector set (i.e. a WH transform) to recover a function, g[i], which matches the original f[i]: g[i]=f[i]. Recovery of the original light pattern involves performing a Walsh-Hadamard transform on the spectrum, which is performed, in one embodiment, according to the following code fragments and starting with the creation of the BW basis set from the WH basis set. Both basis sets are matrices with the basis vectors being the rows of their matrices. Note that the matrix is symmetrical. The WH matrix is also orthogonal, which means that multiplied by itself it produces the identity matrix:
The following code fragment zeros out the negative entries:
Next, we transform f[i] to F[j] by matrix multiplication of f[i] by the BW matrix:
//COMPUTE THE HADAMARD TRANSFORMED FUNCTION F[ ] from f[ ]:
The return from F[j] to an auxiliary function g[i] involves, first, multiplication of F[j] by the WH matrix. Note that the result is rescaled according to the number of elements in the vector, 2/ROW:
//NOW, INVERT THE HADAMARD TRANSFORM TO GET g[ ] (note the rescaling):
The zero term, f[0], then needs adjustment. We make this adjustment by subtracting F[0]:
//Adjust the zero sequency term:
The apparatus 300 also includes a lens assembly 320 located on the second side 324 of the mask 304. The lens assembly 320 projects an image 344 of an object 340 through the mask 304 on to the plurality of light detectors 336 to generate a filtered image. The apparatus 300 also includes an actuator 308 coupled to an image displacing element 314. In this embodiment, the actuator 308 is actuated to move along a single axis 312 parallel to the face 348 of the light detectors 336. The actuator 308 is configured to controllably move the image displacing element 314 along the direction of axis 312. The image displacing element 314 can be, for example, a mobile lens element, a steering mirror or a deformable prism. In this embodiment, the image displacing element 314 is a mobile lens element. The image displacing element 314 serves a dual function in this embodiment. The image displacing element 314 increase the pixel count and also stabilizes the image to compensate for camera shake. The actuator 308 movement can be controlled to account for shaking (undesirable movement) of, for example, the image displacing element 314 due to forces applied to the apparatus 300 by external disturbances.
The apparatus 300 also includes a processor 350 coupled to the actuator 308 and the detector array 332. The processor 350 provides commands to the actuator 308 to controllably move the image displacing element 314. The processor 350 also receives the outputs from the light detectors 336 of the detector array 332. The processor 350 includes computer memory and stores the light detector 336 outputs in the computer memory for subsequent processing and/or retrieval.
The image displacing element 314 moves transversely (along axis 312) to the propagating light. In so doing it displaces the image, along the direction of motion of the lens, with respect to the focal plane.
The image displacing element 314 is repetitively stepped in such a way that each portion of the image falls successively on a different detector 336. Because each detector 336 has a different BW pattern block 316, this repetitive stepping of the image will create a BW sequency analysis of the image for each of the sixteen basis vectors. In order to retrieve the sequency, so the subpixels of the image can be recovered, it is necessary to synchronously retrieve information from the different parts of the memory in which the set of stepped images is stored.
In normal operation, for each stepped image displacing element 314 position, an image is dissected, recorded and stored in memory in the processor 350. Suppose, for example, that a first image is recorded with the image displacing element 314 in a first position; the lens is then stepped to a second position and another image is recorded and stored in a different portion of the memory. We consider a small patch of the image which is the size of a single detector. Because the second image is displaced from the first, this image patch will move to an adjacent detector and will therefore be spatially masked in a different way from the first case. This is because the image patch is now illuminating a detector which has a different BW mask pattern. In addition, this adjacent detector will report its output to a different memory location than the first detector. This process continues until the apparatus 300 has stepped the patch image through all sixteen detector 336 positions.
In order to reconstruct the fine pixel image of this detector-sized image patch, it is necessary to retrieve the patch's intensity measurements made from adjacent detectors as the image is stepped from each detector to an adjacent detector. This sequence of recorded intensities, retrieved from different parts of the computer memory associated with the processor, is then passed through a WH transformation and the desired fine pixel image for the selected image patch is produced.
When the step-and-repeat process has been completed, the stored numbers are withdrawn from memory so as to provide sequency components n through n+3 (for this partial example). From the full set of sequency components (16 in this case) the high resolution image patch can be reconstructed. This reconstructed image patch will then be represented by sixteen micropixels which cover an area the size of a detector.
The apparatus 500 also includes a processor 550 coupled to the actuator 508 and the detector array 532. The processor 550 provides commands to the actuator 508 to controllably move the detector array 532. The processor 550 also receives the outputs from the light detectors 536 of the detector array 532. The processor 550 includes computer memory and stores the light detector 536 outputs in the computer memory for subsequent processing and/or retrieval. The apparatus also includes a lens assembly 520 located on the second side 524 of the mask 504. The lens assembly 504 projects an image 544 of an object 540 through the mask 504 on to the plurality of light detectors 536 to generate a filtered image.
Movement of detector array 532 is performed using a step and repeat motion along the axis 512. The detectors 536 are subjected to the light passing through the mask 504 for a finite amount of time to allow the detectors 536 to accumulate sufficient photo electrons to produce a low noise image from the image 540. Thus, the pattern blocks 516 of the mask 504 are commanded to hold a position for an appropriate length of time while the detector 536 integrates the light. After this short dwell time the detector array 532 is stepped along axis 512. After the detector array 532 has moved sixteen steps (in this example) the motion is reversed and the detector array 532 returns, step wise, to its starting position to repeat the cycle. Two-axis motion may be used in alternative embodiments but would require a different spatial arrangement of the filter patterns 516.
The method also includes moving 612 the mask, plurality of light detectors, or the image in a plane defined by a planar surface of the plurality of light detectors. The step of moving 612 can include moving one of the mask, plurality of light detectors, or the image along one or more axes. In some embodiments, the method includes moving the mask relative to the plurality of light detectors and the image. In other embodiments, the method includes moving the plurality of light detectors relative to the mask and image. In other embodiments, the method includes moving the image relative to the plurality of light detectors and the mask
The method also includes measuring the filtered image 616 projected through the mask with the plurality of light detectors. The method also includes storing a set of data 620 representing the measured filtered image using, for example, the processor 150 of
Comprise, include, and/or plural forms of each are open ended and include the listed parts and can include additional parts that are not listed. And/or is open ended and includes one or more of the listed parts and combinations of the listed parts.
One skilled in the art will realize the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The foregoing embodiments are therefore to be considered in all respects illustrative rather than limiting of the invention described herein. Scope of the invention is thus indicated by the appended claims, rather than by the foregoing description, and all changes that come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.