This invention relates to the qualification of the template patterns for the automated alignment of objects. The patterns for alignment match are the design structures of the objects rather than pre-defined fiducial marks.
Many industrial applications such as electronic assembly and semiconductor manufacturing processes require automatic alignment of objects such as electronic components, printed circuit board or wafers. Most of the prior-art approaches use predefined fiducial marks for alignment. This requires the design and make of the marks on the objects being aligned. This process limits the flexibility of the alignment options and increases system complexity and cost because the marks have to be made on each objects. The mark making process is challenging when fine alignment is required since the variations of the created marks without rigorous control may exceed the required fine precision.
On the other hand, the inherent design patterns of the objects contain structures that could uniquely define the position of the objects and they exist on all objects to be aligned. The fineness of the design patterns naturally matches the alignment accuracy requirement because fine patterns require fine alignment and coarse patterns only require coarse alignment. Therefore, the mark making challenge can be avoided if the design patterns of the object are used as templates directly for alignment purpose without specific design and make of fiducial marks. This removes the extra steps so it could lower the cost and increase the alignment flexibility and accuracy.
The images of design patterns of an object such as circuit board or a region of a wafer could be easily acquired by a camera or other sensors. However, the images could include any customer designed patterns and not all design pattern structures are adequate for alignment. A good alignment template should have unique pattern structures in the alignment coverage region to assure that it will not be confused with other pattern structures within the same region. It also needs to have stable and easily detectable features so that the search algorithm will not miss the correct template location even if the contrast of the pattern image varies. The selection for good templates is challenging regardless whether it is performed manually or by computer automatic selection.
To achieve efficient search, one prior art approaches use multi-resolution templates. A prior art fast multi-resolution automatic template generation and search method is disclosed in Oh and Lee, “Automatic template generation and searching method”, U.S. Pat. No. 6,603,882, Aug. 5, 2003. It generates a multi-resolution template from the input image. The pattern search uses lower resolution results to guide higher resolution search. Wide search ranges are applied only at the lower resolution images and fine-tuning search are performed at higher resolution images. Another prior art approach for efficiently generating templates from design structures by pattern partition and integration is disclosed in Seho Oh, Shih-Jong James Lee, Shinichi Nakajima, Yuji Kokumai, “Partition pattern match and integration method for alignment”, U.S. patent application Ser. No. 10/961,663, Oct. 8, 2004, which is incorporated in its entirety herein.
The automatically generated templates could include a whole image region or decompose a template into a plurality of components as disclosed in Oh and Lee, “Fast invariant matching using template decomposition and synthesis”, U.S. patent application Ser. No. 10/419,913, Apr. 16, 2003. This method decomposes the template into multiple components and performs search by synthesizing the component results.
The focus of the prior art automatic template generation methods is in the fast template generation and efficient pattern search. The resulting templates could support efficient pattern search either from coarse resolution to fine resolution and/or from early components to later components. However, the prior art efficient templates may not contain high quality patterns for good spatial discrimination and variation immunity. The pattern search accuracy and repeatability could be improved if the template quality is improved.
This invention resolves the template quality problem by performing alignment template goodness measurement and qualification for manually selected or automatically generated alignment template(s). The alignment template goodness qualification method of this invention performs measurement and qualification of the signal content, spatial discrimination, and pattern ambiguity of the alignment template(s). If the selected template(s) cannot be qualified, alternative templates could be selected either automatically or manually.
The primary objective of this invention is to qualify the selected template for good alignment outcome. The second objective of this invention is to allow the selection of alternative templates for better alignment outcome. The third objective of the invention is to select good templates to achieve best spatial discrimination. The fourth objective of the invention is to select templates containing good signal content for stable and accurate search result. The fifth objective of the invention is to select good templates with unambiguous patterns for stable and accurate search result. The sixth objective of the invention is to provide quantitative scoring for template signal content. The seventh objective of the invention is to provide quantitative scoring for template spatial discrimination power. The eighth objective of the invention is to provide quantitative scoring for template pattern ambiguity.
An alignment template goodness qualification method receives a pattern image and a pattern based alignment template and performs template goodness measurement using the pattern image and the pattern based alignment template to generate template goodness result output. A template qualification is performed using the template goodness result to generate template qualification result output.
If the template qualification result is acceptable, the pattern based alignment template is outputted as the qualified pattern based alignment template. Otherwise, an alternative template selection is performed using the pattern image, the pattern based alignment template and the template goodness result to generate alternative pattern based alignment template output.
The template goodness measurements include signal content measurement, spatial discrimination measurement and pattern ambiguity measurement.
The preferred embodiment and other aspects of the invention will become apparent from the following detailed description of the invention when read in conjunction with the accompanying drawings, which are provided for the purpose of describing embodiments of the invention and not for limiting same, in which:
I. Application Scenario
II. Template Goodness Measurement
The template goodness measurement 116 method inputs a pattern image 100 and a pattern based alignment template 102. It uses the input data to generate at least one or a plurality of template goodness results 104. In one embodiment of the invention, a spatial discrimination measurement is included in the template goodness measurement to generate at least one spatial discrimination score for the template goodness result. In another embodiment of the invention, a pattern ambiguity measurement is included in the template goodness measurement to generate at least one pattern ambiguity score for the template goodness result. In yet another embodiment of the invention, a signal content measurement is included in the template goodness measurement to generate at least one signal score for the template goodness results. Those skilled in the art should recognize that different measurements could be combined to yield comprehensive template goodness results.
II.1 Signal Content Measurement
Given a pattern image 100 and a pattern based alignment template 102, two signal scores are generated in one embodiment of the invention. The two signal scores include a region signal score that calculates the signal content for the template generation region and a template signal score that calculates the signal content for the selected template region. Those skilled in the art should recognize that one or both signal scores could be measured depending on the complexity of an application.
A. Region Signal Content Measurement
The region signal content measurement calculates the signal score for the signal measurement region. In one embodiment of the invention, template generation region is used for signal measurement region. The template generation region is the region that is available for the template(s) to be selected. In one embodiment of the invention, the template generation region is the pattern image 100. In another embodiment of the invention, the template generation region is the region that is expanded from the template region. The expansion could be performed by morphological dilation of the template region mask.
Given a signal measurement region (such as the template generation region), I_r, its region signal score (region signal content measurement) is derived from the image pattern structure features contained in the region. In one embodiment of the invention, the image structure features are enhanced using the structure guided image feature enhancement method disclosed in Shih-Jong J. Lee, “Structure-guided image processing and image feature enhancement”, U.S. Pat. No. 6,463,175, October, 2002. The structure-guided image feature enhancement method uses two-dimensional, full grayscale processing and can be implemented efficiently and cost-effectively. The processing is nonlinear and therefore does not introduce phase shift and/or blurry effect. In one embodiment of invention the relevant structure features used including bright edge, dark edge, bright line or region, dark line or region and region contrast.
Bright Edge Enhancement:
Bright edges can be enhanced by a grayscale erosion residue processing sequence defined by:
I-IΘA
Where I is an input image and A is a structuring element and Θ is the grayscale morphological erosion operation.
Dark Edge Enhancement:
Dark edges can be enhanced by a grayscale dilation residue processing sequence defined by:
I⊕A-I
Where ⊕0 is the grayscale morphological dilation operation.
Bright Line or Region Enhancement:
Bright line or region can be enhanced by a grayscale opening residue processing sequence defined by:
I-IOA
Where O is the grayscale morphological opening operation.
Dark Line or Region Enhancement:
Dark line or region can be enhanced by a grayscale closing residue processing sequence defined by:
I●A-I
Where ● is the grayscale morphological closing operation.
Region Contrast Enhancement:
Region contrast can be enhanced by the difference of grayscale closing and opening. The processing sequence is defined by:
I●A-IOA
In one embodiment of the invention, the proportion of the signal pixels within the signal measurement region is calculated as the signal score. That is,
The structure features are enhanced for the pixels in the signal measurement region. The structure for enhancement could be edge, line or region, contrast, or other linear or nonlinear processing to highlight structures of the region. In one embodiment of the invention, the signal pixels are the pixels within the signal measurement region whose enhanced structure feature values are higher than a threshold, T_h. That is,
Signal(x,y)=1 if F(x,y)>T—h. Otherwise, Signal(x,y)=0.
The threshold value, T_h, could be determined as a function of μ_f, the average value of the structure feature enhanced values within the signal measurement region. In one embodiment of the invention, the T_h is calculated as:
T—h=K*μ—f where K>1.0 or
T—h=μf+H where H is either a fixed value a function of the feature value distribution (such as the standard deviation).
In another embodiment of the invention, the signal score is derived from the feature enhancement region statistics. It can be determined as the value corresponding to a certain percentile of the feature enhancement region pixels. That is,
Signal_score=enhance—p(I—r).
Where enhance_p(I_r) is the p percentile value of the feature enhanced pixel values in region I_r. The feature enhancement could be the edge enhancement, line or region enhancement, contrast enhancement, or other linear or nonlinear processing to highlight structures of the region
In yet another embodiment of the invention, the signal score is derived from a combination of the statistics derived from coarse and fine feature enhancements of the region. In this embodiment of the invention, the formula for signal score calculation is as follows:
Signal score=MIN(Fine—80%, Coarse—99%/3.0).
Where Coarse—99% is the 99 percentile (close to maximum) pixel value of the I_coarse_enhance and Fine—80% is the 80 percentile pixel value of the I_fine_enhance. The I_coarse_enhance is the coarse feature enhanced signal measurement region. The I_fine_enhance is the fine feature enhanced signal measurement region. The coarse feature enhancement uses larger structuring element than the coarse feature enhancement. In one embodiment of the invention, the contrast feature is used for the enhancement and the contrast enhancement is performed as follows:
I_coarse_enhance=I—r●9×9−I—r O 9×9
I_fine_enhance=(I—r*5×5−I—r O 5×5)⊕5×5
Where
9×9 designates a flat top morphological structuring element of size 9 pixels by 9 pixels;
5×5 designates a flat top morphological structuring element of size 5 pixels by 5 pixels.
Those skilled in the art should recognize that other feature enhancement methods such as the edge enhancement, line or region enhancement, or other linear or nonlinear processing to highlight structures of the region can be similarly applied. Also, the size s of the structuring elements and the percentiles (99% and 80%) used could be changed. In addition, the weighting factor (1/3.0) could also be changed.
B. Template Signal Content Measurement
The template signal content measurement calculates the signal score for a template region. In one embodiment of the invention, at least one directional signal is calculated. The direction can be vertical, horizontal, diagonal, or any given arbitrary directions. The directional signal is measured using directional projection and signal range derivation method. The scores for vertical and horizontal signals are described in this section. The vertical signal score measures the vertical structure signal content within the template region. The horizontal signal score measures the horizontal structure signal content within the template region. Those skilled in the art should recognize that the scope of the invention should cover any directions rather than limited to vertical and horizontal directions.
Vertical Signal Score
In one embodiment of the invention, given a template component C having width W and height H, its vertical signal score, Vertical_signal_C, can be calculated by the following procedures:
Those skilled in the art should recognize that maximum value could be replaced by an upper percentile value; minimum value could be replaced by a lower percentile value; median value could be replaced by other data center estimator (such as mean) for the signal range and signal score calculations. Also, the region can be divided into one, three or more sub-regions rather than two halves for projection and signal range deviation measurement.
Those skilled in the art should recognize that the combination of Vertical_signal_T_C and Vertical_signal_B_C to create Vertical_signal_C, could be done by other means such as linear combination or multiplication, etc.
Horizontal Signal Score
Similarly, in one embodiment of the invention, given a template component C having width W and height H, its horizontal signal score, Horizontal_signal_C, can be calculated by the following procedures:
Those skilled in the art should recognize that maximum value could be replaced by an upper percentile value; minimum value could be replaced by a lower percentile value; median value could be replaced by other data center estimator (such as mean) for the signal range and signal score calculations. Also, the region can be divided into one, three or more sub-regions rather than two halves for projection and signal range deviation measurement.
Those skilled in the art should recognize that the combination of Horizontal_signal_L_C and Horizontal_signal_R_C to create Horizontal_signal_C, could be done by other means such as linear combination or multiplication, etc.
II.2 Spatial Discrimination Measurement
Given a pattern image and a pattern based alignment template the spatial discrimination measurement method measures the spatial discrimination scores. A good alignment template should be able to uniquely define a matching position for alignment.
In the case that the template has only one component (region), the spatial discrimination (unique position) has to be achieved within the component. In the case that a plurality of template components exist, the spatial discrimination could be achieved by the combination of the plurality of template components. For example, one component could provide a unique X position and the other component could provide a unique Y position.
The spatial discrimination measurement for a two component template is illustrated in
In one embodiment of the invention, the spatial discrimination score is derived from combinations of at least two directional signal scores (such as the vertical and horizontal signal scores) and the region signal score of the template generation region.
II.2.1 Raw Discrimination Score
The template (combination of all components) vertical signal score is defined as the maximum vertical signal score between its component Ci where i≧1 as follows:
Similarly, the template horizontal signal score is defined as the maximum horizontal signal score between its component Ci where i≧1 as follows:
The raw discrimination score is defined as the minimum of Vertical_signal and Horizontal_signal:
Raw_discrimination=MIN(Vertical_signal, Horizontal_signal)
That is, the raw discrimination score of a template is the worst of its template vertical signal score and template horizontal signal score. This enforces the requirement that a good template discrimination needs to have good signals in both vertical and horizontal directions.
II.2.2 Component Discrimination Score
The discrimination score could be determined for each component Ci of the template as follows:
Discrimination—Ci=MIN(Vertical_signal—Ci, Horizontal_signal—Ci)
where i≧1.
An integrated component discrimination score can be defined as follows:
Even though Discrimination_Ci is less than or equal to Raw_discrimination, the Integrated_component_discrimination could be greater than Raw_discrimination if the summation of the integration factor αi for all i is greater than 1.0. The Integrated_component_discrimination has a higher value than Raw_discrimination when many of the Discrimination_Ci have high values. That is, many components have good patterns to define both X and Y positions.
II.2.3 Spatial Discrimination Score
In one embodiment of the invention, a combined discrimination score, Combined_discrimination, is defined as:
Combined_discrimination=MAX(Raw_discrimination, Integrated_component_discrimination).
A normalized combined discrimination score, Normalized_combined_discrimination, that normalizes the combined discrimination score by the-region signal score is defined as:
Normalized_combined_discrimination=Combined_discrimination/Signal_score
Finally, the spatial discrimination score, spatial_discrimination_score, is defined as a function of the Combined_discrimination and the Normalized_combined_discrimination. In one embodiment of the invention, a quadratic combination is used:
Spatial_discrimination_score=K1*(Combined_discrimination)2+K2*(Normalized_combined_discrimination)2
Where K1 and K2 are the weighting factor for the quadratic combination.
Those skilled in the art should recognize that other method of combination such as linear, polynomial, geometric mean, etc. Furthermore, the Combined_discrimination or Normalized_combined_discrimination can be used as the Spatial_discrimination_score without the combination.
The Spatial_discrimination_score is the summary score for the spatial discrimination power of the template. Higher Spatial_discrimination_score value corresponds to better template spatial discrimination power.
Those skilled in the art should recognize that even though the spatial discrimination score is derived from measurements of signals in the horizontal and vertical directions only. The signals of other directions can also be used for the spatial discrimination score. Furthermore, the spatial discrimination score can be performed for single component template or for templates with two or more components.
II.3 Pattern Ambiguity Measurement
Given a pattern image and a pattern based alignment template, the pattern ambiguity measurement measures the ambiguity of the template patterns around the neighbor of the template.
In one embodiment of the invention, the pattern ambiguity score is calculated as
Pattern ambiguity score=M1/M
Where M is the auto-matching value of the template region and M1 is the maximum matching value between the template and the image pixels within the ambiguity check region, which is the neighbor of the template. The neighbor can be determined by dilating the template region by a structuring element of a desired neighboring size. The matching value can be determined by the normalized correlation (Ballard D H and Brown C M, “Computer Vision”, Prentice-Hall Inc. 1982 pp. 68-70). Other matching methods could also be used such as absolute difference, simple image multiplication, etc.
III. Template Qualification
The template qualification method checks the template goodness results to determine whether the template is acceptable or not. In one embodiment of the invention, the template qualification performs signal content qualification check that rejects a template if its signal score is less than a threshold. In another embodiment of the invention, the template qualification performs spatial discrimination qualification check that rejects a template if its spatial discrimination score is less than a threshold. In yet another embodiment of the invention, the template qualification performs pattern ambiguity qualification check that rejects a template if its pattern ambiguity score is greater than a threshold.
The template qualification can also be applied to a plurality of the scores simultaneously by combining the scores. In one embodiment of the invention, a weighted linear combination is applied to the signal score, spatial discrimination score, and the inverse of the pattern ambiguity score to generate an integrated score. The weighting factors for the scores normalize the scores into similar ranges and account for the individual variations of the scores. A threshold can be applied to the integrated score. A template is rejected if its integrated score is less than a threshold.
Those skilled in the art should recognize that other methods of the score combination such as polynomial combination, multiplication, logarithm, square root, or other nonlinear combination can also be used.
The invention has been described herein in considerable detail in order to comply with the Patent Statutes and to provide those skilled in the art with the information needed to apply the novel principles and to construct and use such specialized components as are required. However, it is to be understood that the inventions can be carried out by specifically different equipment and devices, and that various modifications, both as to the equipment details and operating procedures, can be accomplished without departing from the scope of the invention itself.