Patent Application
20120206634
The present invention relates generally to optical scanning systems and sensors, and specifically to scanning techniques employing two dimensional sensor arrays.
Scanner systems for acquiring an image of an object, such as a printed circuit board, are well known in the arts of imaging and automated optical inspection. Some conventional scanner systems include sensors comprising a linear array of sensor elements. Other conventional scanner systems include sensors comprising a two dimensional array of sensor elements. Some systems employing two-dimensional array of sensor elements have been configured, for example, to operate in a time delay integration (TDI) mode of operation to acquire and image of an object. Other system employing a two-dimensional array of sensor elements have been configured to acquire a sequence of non overlapping images of an object.
TDI systems are well known in the art of optical scanning. In such systems, a sensor array is scanned over an object, such as a printed circuit board, by moving either the array or the object in a direction perpendicular to the rows of the array. The scanning speed and an array clock are synchronized so that in each column of the array, multiple sensor elements in sequence capture light from the same point on the object. Charge is accumulated between rows as the sensor array passes over the object so that sensor signals in each column are summed for each point on the object, thereby providing an image of the object with enhanced signal/noise ratio.
Common TDI sensors are based on charge-coupled device (CCD) technology, which allows the sensor signals to be summed by transferring charge along each column of the sensory array such that newly accumulated charge is added to charge having accumulated in previous rows of the column. Other TDI systems based on photodiode arrays, such as CMOS sensor arrays, are also known in the art.
U.S. Pat. No. 5,750,985 and U.S. Pat. No. 5,909,026, the disclosures of which are incorporated by reference, both describe sensors that can be employed in a TDI type arrangement.
Applicants' copending U.S. patent application Ser. No. 10/141,988, filed on May 10, 2002 and entitled “Optical Inspection System Employing a Staring Array Scanner”, the disclosure of which is incorporated by reference, describes an inspection system employing a two dimensional sensor array.
It is an object of some aspects of the present invention to provide improved systems and methods for automated optical inspection (AOI).
It is a further object of some aspects of the present invention to provide improved imaging techniques and devices employing two dimensional sensors.
In accordance with a broad aspect of the present invention, an at least two dimensional array of photosensors formed on a CMOS integrated circuit is employed to acquire images representing an object, such as images of an electrical circuit. At least partially overlapping images are acquired, and pixels in the overlapping images, associated with corresponding portions of the object, are added together to form a composite image of the object. The composite image is particularly useful, for example, to inspect the object for defects. As used herein the term CMOS integrated circuit generally includes any suitable integrated circuit comprising photosensors, such as photodiodes or photogates, other than CCD type photosensors.
In preferred embodiments of the present invention, a two-dimensional imaging device comprises a two-dimensional sensor array and a memory, having cells arranged in rows and columns that correspond to the rows and columns of sensor elements in the array. The array and memory are configured to operate in a memory integration mode so as to provide a composite image as the array scans over an object. In each cycle of the array clock (i.e., each time the sensor array captures an image frame), the signal received by each of the sensor elements is digitized and added to the value stored in one of the cells of the memory. A dynamic input pointer indicates, for each row of the sensor array, the row in the memory into which the signals from the sensor elements in that row of the array should be added. The input pointer is advanced at each cycle of the array clock in such a way that each memory cell receives a sum of signals from multiple sensors in the same column of the array, captured as the sensors pass over the same point on the object. A dynamic output pointer is also updated at each cycle to indicate the row of the memory in which integration has been completed, so that the memory integrated signal can be read out.
This use of dynamic input and output pointers enables the sensor array, memory and memory integration logic to be efficiently implemented together on a single chip, preferably using active pixel CMOS sensor elements. The dynamic pointer scheme also allows the direction of scanning the array to be reversed simply by reversing the pointer direction, so that the object can be scanned in a bidirectional serpentine pattern, for example. This feature is particularly useful in AOI systems.
In some preferred embodiments of the present invention, the sensor array comprises color filters, so that the memory integrated image captured by the array and memory comprises a color image. Preferably, the color filters are arranged so that successive rows of the array receive light of different colors, typically in a repeating red-green-blue pattern. Alternatively, the color filters may be arranged so that successive groups of rows receive light of different colors, for example, several red rows, followed by several green rows, followed by several blue rows. Dynamic input and output pointers are provided for each color. The dynamic pointer configuration and scan rate of the array over the object may be chosen to give either a color memory integrated image with full resolution, equal to that of a comparable monochrome memory integrated image, or a color memory integrated image that has reduced resolution, but whose throughput (i.e., speed of image capture) is equal to that of a monochrome memory integrated image.
In some preferred embodiments of the present invention, the two dimensional imager is used in an AOI system, typically for evaluating characteristics of objects such as printed circuit boards, flat panel displays, electronic assembly boards, and the like. The features of the memory integrated imager described above enable the system to operate at high speed and with high sensitivity, in either a monochrome or color imaging mode.
The present invention will be more fully understood from the following detailed description of the preferred embodiments thereof, taken together with the drawings in which:
System 20 captures images of board 22 using a camera 24, which is built around a CMOS integrated circuit imaging device 26 having an at least two dimensional array of photosensors integrally formed thereon. In accordance with an embodiment of the invention, imaging device 26 is operational in a memory integration mode of operation. An objective lens 28 forms an image of board 22 on device 26 as camera 24 is scanned over the surface of the board by a translation stage 32. Preferably, the camera is scanned over the surface in a bidirectional serpentine pattern, so that the entire surface is imaged by the camera at a desired level of resolution. Alternatively, board 22 may be translated while holding camera 24 still, or both the board and camera may be translated, typically in mutually-perpendicular directions. A light source (not shown) illuminates board 22 as it is imaged by camera 24, preferably by providing generally continuous illumination, or by providing non-continuous illumination that is generally synchronized with a frame rate of image frames acquired by imaging device 26.
A camera control unit 30 regulates the timing and operation of device 26, and passes image data from device 26 to an image processor, or analyzer, 34. The image processor analyzes the image data to locate and identify faults, or defects, in board 22. In accordance with a preferred embodiment of the invention processor 34 comprises combinations of image processing hardware and software such as are used in various AOI systems available from Orbotech Ltd. of Yavne, Israel, including the Inspire 9060™ and SK-75™ AOI systems. Alternatively or additionally, processor 34 may comprise a general-purpose computer with suitable input circuits and software for this purpose, hard-wired logic and/or a programmable digital signal processor. For each board tested by system 20, processor 34 outputs either a notification that the board is acceptable or an indication (such as a map) of a fault or faults found in the board, via a display 36 or other output interface.
As seen in
In accordance with an embodiment of the invention, images 240-248 are digital pixel images that are sequentially acquired by imaging device 26 during scanning a portion of board 22. Only five images are shown for the sake of simplicity. Typically a much greater number of images is acquired. Each of the images 240-248 corresponds to a mutually offset portion of board 22 such that each image of board 22 acquired by imaging device 26 at least partially overlaps another image. The mutual offset between images may be as small as 1 pixel, although the mutual offset between images may be greater.
Thus, as seen in
A composite image generator 252 is operative to combine together each of the partially overlapping images generated by array 40, for example images 240-248, and to supply composite image data 260 to analyzer 234. The composite image data 260 forms image 220 which has an improved signal/noise ratio compared to images 240-248. Image 220 is used by defect analyzer 234 to detect defects in board 22.
In accordance with an embodiment of the invention, composite image generator 252 is integrally formed on imaging device 26, although this need not be the case. As seen in
The operation of composite image generator may be better understood from
Pixels in image 244 are added to corresponding pixels in first composite image 274 to result in second composite image 278. It is seen that image 244 is offset relative to first composite image 274 and includes a sequentially added row of pixels 280. Pixels to the left of row 276 in second composite image 278 are darker than pixels in row 276, and pixels in row 276 are darker than pixels in row 280.
Pixels in image 246 are added to corresponding pixels in second composite image 278 to result in third composite image 282. It is seen that image 246 is offset relative to second composite image 278 and includes a sequentially added row of pixels 284. Pixels to the left of row 276 in third composite image 282 are darker than pixels in row 276, pixels in row 276 are darker than pixels in row 280, and pixels in row 280 are darker than pixels in row 284.
Pixels in image 248 are added to corresponding pixels in third composite image 282 to result in fourth composite image 286. It is seen that image 248 is offset relative to third composite image 282 and includes a sequentially added row of pixels 288. Pixels to the left of row 276 in fourth composite image 286 are darker than pixels in row 276, pixels in row 276 are darker than pixels in row 280, pixels in row 280 are darker than pixels in row 284, and pixels in row 284 are darker than pixels in row 288.
The above process is continued sequentially until a desired quantity of corresponding pixels are added together such that a gradient is formed in the composite image. At the end of each frame, line of pixels comprising the result of adding together a plurality of pixels, is provided as image data 260 (
It is a feature of some embodiments of the present invention that values added together in the respective images 240-248 are digital values. The digital values are provided by at least one A/D converter associated with photosensors 42. An A/D converter may be associated with each photosensor 42. Optionally, each A/D converter is associated with a plurality of photosensors 42. For example, each A/D converter is associated with a row of photosensors.
Preferred embodiments of the architecture, functionality and operation of imaging device 26 will be discussed hereinbelow in greater detail. In general, it is noted that imaging device 26 includes a plurality of digital registers which are operative to temporarily store the outputs of the A/D converters, digital memory, typically including an array of memory cells, storing image data provided by the array of photosensors, and a plurality of digital adders operative to add the outputs of the digital registers to corresponding image data which is stored in the digital memory.
Moreover, in accordance with embodiments of the invention the adding together of images, such as images 240-248, is performed on the fly on a line by line basis, and composite images are stored in a memory array in a wrap-around manner that dynamically changes as each new image 240-248 is acquired and added to a previously stored composite image.
It is noted that images 240-248 seen in
The digitized signal values held in register array 46 are summed by an array 48 of adders with corresponding stored values in rows of a memory 50, which typically comprises high-speed static or dynamic random access memory (SRAM or DRAM) or any other suitable type of memory. The results of the summation are stored back in the same row of the memory. This read/sum/store operation is typically performed for each cell in memory 50 once per frame. It is repeated over a predetermined number of frames, each time adding in the signal from a different row in array 40, until the memory cell contains the sum of the signals taken from the predetermined number of different elements 42 in the same column of the array. The association of sensor elements with memory cells at each cycle is controlled by a system of dynamic pointers, as described below. After the required number of summations of values from different elements 42 have been performed for a given row of memory 50, the results in that row are read out to an array 52 of output registers. These data are then clocked out of the registers to output ports 56 via a column decoder 54, for readout to processor 34.
In a preferred embodiment of the invention, as seen in
Each sensor element 42 comprises a photodetector (or photosensor) 70, typically a photodiode or photogate, and an active amplifier 72 which also includes, for example, a select transistor (not shown). The amplifiers are triggered by row select lines 74 to read out the charge stored by the corresponding photodetectors to column output lines 76. Photodetectors 70 are preferably designed for low capacitance, in order to reduce the level of reset thermal (kTC) noise that they generate. In accordance with a preferred embodiment, each pixel also comprises a reset circuitry (not shown), which is separately controlled by reset control 62. Optionally, each sensor element may comprise a separate charge storage element, such as a capacitor (not shown), to which charge is transferred from the photodetector and held until it is read out of the array. As a further option, mentioned above, each sensor element may comprise a built-in A/D converter (not shown). Other means known in the art may also be used to enhance the sensitivity and signal/noise ratio of array 40, such as the use of microlenses, integrated with the array, to focus light received by camera 24 onto photodetector 70 within each sensor element 42.
Preferably, A/D converter array 44 comprises one A/D converter 78 per column of array 40. Optionally, an A/D converter may be associated with each element 42. At each cycle of the row clock generated by row timing block 58, converter 78 digitizes the signal from a successive element 42 in its corresponding column of array 40. The digitized value is held in a register 80 in register array 46, until it is summed by an adder 82 with the contents of a selected cell 84 in memory 50. The sums output by adders 82 are written back to the same cells in memory 50 from which the addends were read out. Memory cells 84 are arranged in columns corresponding to the columns of sensor elements 42 in array 40. In the present embodiment, cells 84 are arranged in four rows 86, corresponding to the four rows of elements 42 in array 40. The row whose cells 84 are to be read out for summing by adders 82 at each cycle of the row clock is determined by an input pointer 88. After a complete frame has been read out of array 40, digitized and summed into the appropriate cells in memory 50, pointer 88 is advanced to a new position for the next frame. As a result, each cell 84 in memory 50 receives the sum of the signals generated by all four sensor elements 42 in the corresponding column of array 40.
An output pointer 92 is used to indicate the row 86 in memory 50 whose cells 84 contain the summed signals from all four of the sensor elements 42 in the corresponding column of array 40. At each cycle of the row clock, the contents of these cells are read out to registers 90 in output register array 52. After the contents of a row of cells have been read out, the cells are reset to zero. Then, during the next frame, input pointer 88 is advanced so that the null contents of these memory cells are summed with the signals from the sensor elements in the first row of sensor array 40. In each subsequent frame, the pointers are advanced, and summations are performed, until the cells again contain the sum of signals from all four rows of the sensor array and can again be read out. Output pointer 92 is likewise advanced in each frame to point to the next row of memory 50 that is to be read out.
Input pointer 88 is set in each frame to point to the cell 84 in memory 50 to which the signal from sensor element 42a is to be added. The location of the input pointer in each of the four successive frames (corresponding to bars 100a-d) is shown in
Output pointer 92 is set in each frame to point to the cell 84 in memory 50 to which the signal from sensor element 42d is added. This cell will contain, after the output of adder 82 is written back to the cell, the sum of the signals from all four of sensor elements 42a-42d in column 102 for each successive pixel on object 100. The contents of this cell can thus be read out to register 90 and then reset to zero. The position of the output pointer in each of the four successive frames is shown in
After all the summations are complete, the summed data are read out of cells 84 in the row 86 of memory 50 that is indicated by output pointer 92. Pointers 88 and 92 are then advanced to their positions for the next frame. As soon as all the rows of array 40 have been read out (even before the pointers are advanced), the array can be reset, and the process begun over again.
Note that because of the order of reading out rows 1 through N of array 40, the integration times of the rows are not uniform. Row 1 has the shortest integration time, while row N has the longest. (The timing pattern shown in
Although the embodiments described up to now are directed to monochrome imaging, system 20 and imaging device 26 may also be adapted to capture color images of board 22. One approach for this purpose would be to use colored strobe illumination (not shown), for example synchronized with the array clock, in which a different color light (typically red, green or blue) is used to illuminate the board in each successive frame, or for several successive frames. In order to generate color images, memory 50 must be divided into separate sections, for receiving and integrating the signals corresponding to the different colors. Within each section, the data are summed and read out using input and output pointers in substantially the same way as described above. As another alternative, described below with reference to the figures that follow, different color filters are applied to separate rows of array 40. Of course, different color filters may also be applied to separate columns of the sensor array, but this option may be less desirable as it necessarily detracts from the resolution of camera 24.
In the preferred embodiment seen in
For each of stages I through IV, the figure shows the location of input pointers 116 and output pointers 118, along with the summed signals held in each memory cell 84. Three input pointers and three output pointers are provided, one for each color group. At each stage, the signals from the first red, green and blue pixels (R1, G1 and B1) are read into the memory cells indicated by the respective input pointers 116. The signals from the remaining pixels in each color group are summed into the next memory cells in column 104, in a manner similar to that shown in
It will be observed that the red, green and blue outputs generated by array 106 are out of registration by three rows (amounting to six rows between the red and the blue outputs). The registration can be easily corrected by adding a six-stage buffer for the red output and a three-stage buffer for the green output. These buffers can be provided in the two-dimensional color scanning imaging device itself or on a separate chip in camera 24. Alternatively, the registration adjustment may be performed by processor 34 without prior buffering.
As in the preceding embodiment, three input pointers 116 are provided, indicating cells 84 to which the first red, green and blue sensor signals (R1, G1 and B1) are to be written at each stage. Three output pointers 118 indicate the cells from which the summed pixel values are to be read out. For the present embodiment, in which the RGB cycle repeats four times, the pointers return to their starting values after four frames, labeled stages 1, 2, 3 and 4, are completed.
The memory in the embodiment of
At each stage, output pointers 118 indicate three adjacent memory cells 84 to be read out from one of the three sections. The output pointers alternate from section to section in each cycle. The three cells that are read out in each stage contain the red, green and blue pixel values, respectively, of one of the pixels, which are read out of the memory simultaneously. Since buffering is performed in the memory itself, no external buffering is required. Table II below lists the pixels whose values are read out at each stage:
Although the preferred embodiments described above relate particularly to detection of visible light, the principles of the present invention may similarly be adapted for detection of other types of radiation, and particularly for infrared and ultraviolet light. Thus, the “colors” mentioned above should be interpreted more generally as referring to different wavelength bands.
It will thus be appreciated that the preferred embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.
The present application is a continuation of U.S. patent application Ser. No. 13/013,967 filed on Jan. 26, 2011, which is a continuation of U.S. patent application Ser. No. 12/179,033, filed on Jul. 24, 2008, which is a divisional of U.S. patent application Ser. No. 11/532,549, filed on Sep. 18, 2006, issued as U.S. Pat. No. 7,417,243, on Aug. 26, 2008, which is a divisional of U.S. patent application Ser. No. 11/225,041, filed on Sep. 14, 2005, issued as U.S. Pat. No. 7,129,509, on Oct. 31, 2006, which is a continuation of application Ser. No. 10/176,003, filed on Jun. 21, 2002, issued as U.S. Pat. No. 7,009,163, on Mar. 7, 2006, which is a Non-Provisional of U.S. Provisional Patent Application No. 60/299,766, filed Jun. 22, 2001. The entire disclosures of the prior applications are hereby incorporated by reference in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 60299766 | Jun 2001 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 11225041 | Sep 2005 | US |
| Child | 11532549 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13013967 | Jan 2011 | US |
| Child | 13351542 | US | |
| Parent | 12179033 | Jul 2008 | US |
| Child | 13013967 | US | |
| Parent | 11532549 | Sep 2006 | US |
| Child | 12179033 | US | |
| Parent | 10176003 | Jun 2002 | US |
| Child | 11225041 | US |
