The present invention relates to a technique for recognizing and detecting objects, such as fiducial mark and land, disposed on flat surfaces, such as printed circuit board and screen printing mask, based on pattern matching for positioning, measurement, and inspection.
Conventionally, pattern matching using the gray-scale normalized correlation method is often applied to recognition or detection of different objects of regular pattern in various images. This technique for matching a pattern image, i.e. a reference image for pattern matching with a corresponding input image, i.e. an image to be recognized or detected overlays the pattern image on the input image enclosed in a search frame to carry out product sum calculation for each corresponding pixel, obtaining a normalized correlation coefficient value. When calculating the normalized correlation coefficient value, the pattern image is shifted little by little with respect to the input image enclosed in the search frame to obtain its maximum value, i.e. repeated calculation processing is needed for the entire input image.
Conventional pattern matching using the normalized correlation method arises the following inconveniences:
1) With measuring apparatus based on pattern matching, fiducial marks, lands, etc. of solder levelers suffer a nonuniform density due to diffused reflection on their surface irregularities even with images subjected to image processing. Thus, when matching a pattern image derived from an original image with a corresponding input image, two images patterns with different nonuniform densities may cause lowered conformity of the density distribution therebetween to reduce a normalized correlation coefficient value, resulting in unsuccessful pattern matching.
Further, when matching an image of regular pattern with a corresponding input image, a difference in density distribution of the two images, i.e. insufficient similarity of the density distribution therebetween makes pattern matching unsuccessful. This raises a problem of difficult positional detection of a fiducial mark and a land of a solder leveler as shown in
Furthermore, with conventional pattern matching, when an input image enclosed in a search frame is scanned by a pattern image, product sum processing is repeatedly carried out to obtain a particular normalized correlation coefficient value between the pattern image and the input image, necessitating enormous calculation processing time. Thus, a computer cannot carry out real time processing only with its central processing unit (CPU), requiring an exclusive fast image processing board, resulting not only in restricted development of a specific algorithm, but in increased manufacturing cost due to complex apparatus structure.
2) Conventionally, in order to detect character such as lot number, original images of the characters are previously stored as pattern images, and are subjected to pattern matching with corresponding input images of the characters. According to this method, if a substrate has shading, circuit sign drawn by silk-white paint (refer hereafter to as silk-white circuit sign), local dirt and the like on the background of an image of a character, a normalized correlation coefficient value between the pattern image and the input image is lowered considerably to disallow detection of the character.
Referring to
Even with some disturbances produced on the background on an input image, human eyes can recognize a character based on feature information of the foreground of the image. In order to allow detection of characters regardless of disturbed background, a novel method is demanded which ensures pattern matching with regard to only the foreground with the background excluded.
3) For detection of lands, solders, mounted parts and the like of regular pattern, their original images are stored as pattern images, and are subjected to pattern matching with corresponding input images.
If disturbance factors such as wiring layout pattern, silk-white circuit sign, resist and flux exist on the background of pattern images in the vicinity of the lands, solders, etc., the background suffers a nonuniform density or nonuniform and complex density distribution. It is noted that the wiring layout pattern can produce a complex density distribution, and the silk-white circuit sign can cause a greater density peak on the background than that on the foreground. Resist and flux can cause a nonuniform density and variations in density level on the background.
Thus, due to unsuccessful pattern matching on the background, a normalized correlation coefficient value between a pattern image and a corresponding input image is lowered to make pattern detection unsuccessful.
It is, therefore, an object of the present invention to provide a method of recognizing an object based on pattern matching, which allows stable and sure pattern detection with largely shortened processing time, simplified structure, and reduced manufacturing cost.
Another object of the present invention is to provide a medium for recording a computer program having the inventive method.
Generally, the present invention provides a method of recognizing an object based on pattern matching using a gray-scale normalized correlation method, comprising the steps of:
storing a reference image including a foreground and a background, said foreground and said background each having a predetermined value of density distribution;
inputting an image of the object, said image including a foreground and a background, said foreground and said background each having a predetermined average value of density distribution;
storing a function for giving said predetermined values of density distribution of said reference image equal to said predetermined average values of density distribution of said input image, respectively; and
obtaining a maximum normalized correlation coefficient between said reference image and said input image using said function.
An aspect of the present invention is to provide a method of recognizing an object based on pattern matching using a gray-scale normalized correlation method, comprising the steps of:
storing a reference image including a foreground, said foreground having a predetermined value of density distribution;
inputting an image of the object, said image including a foreground, said foreground having a predetermined average value of density distribution;
storing a function for giving said predetermined value of density distribution of said reference image equal to said predetermined average value of density distribution of said input image; and
obtaining a maximum normalized correlation coefficient between said reference image and said input image using said function.
Another aspect of the present invention is to provide a medium for recording a computer program having a method of recognizing an object based on pattern matching using a gray-scale normalized correlation method, the method comprising the steps of:
storing a reference image including a foreground and a background, said foreground and said background each having a predetermined value of density distribution;
inputting an image of the object, said image including a foreground and a background, said foreground and said background each having a predetermined average value of density distribution;
storing a function for giving said predetermined values of density distribution of said reference image equal to said predetermined average values of density distribution of said input image, respectively; and
obtaining a maximum normalized correlation coefficient between said reference image and said input image using said function.
A further aspect of the present invention is to provide a medium for recording a computer program having a method of recognizing an object based on pattern matching using a gray-scale normalized correlation method, comprising the steps of:
storing a reference image including a foreground, said foreground having a predetermined value of density distribution;
inputting an image of the object, said image including a foreground, said foreground having a predetermined average value of density distribution;
storing a function for giving said predetermined value of density distribution of said reference image equal to said predetermined average value of density distribution of said input image; and
obtaining a maximum normalized correlation coefficient between said reference image and said input image using said function.
Referring to the drawings, preferred embodiments of the present invention will be described in detail.
An essential factor for increasing the detection ability of pattern matching comprises registration of pattern images. The shape of a pattern image as well as the density distribution of the foreground and background thereof is the key for increasing conformity of pattern matching between a pattern image and a corresponding input image.
According to the equation of definition of the normalized correlation coefficient Cr, achievement of high conformity (Cr=I) requires that the shape and area or size of the foreground of a pattern image, i.e. a reference image for pattern matching are substantially equal to those of the foreground of a corresponding input image, i.e. an image to be recognized or detected. In this connection, refer to the relation between images as shown in
With regard to the density distribution, only similarity is needed between the foreground and the background of a pattern image and those of a corresponding input image. In this connection, refer to the relation between graphical forms as shown in
If density noise occurs due to diffused reflection on a solder leveler, for example two images patterns with different nonuniform densities of a pattern image and a corresponding input image may cause lowered similarity of the density distribution therebetween, resulting in unsuccessful pattern matching.
Then, in order to smooth noise of the foreground and background of the pattern image, average densities of the foreground and background of the pattern image are set in connection with the density distribution, respectively. A density distribution function using the average densities takes a higher constant value in the domain of the foreground, and a lower constant value in the domain of the background, providing a function of plateau-like or rectangular section form as shown in 8B. This function is suitable for models of fiducial marks and lands having regular shape, constant surface properties, and uniform density in nature.
Alternatively, the density distribution function may take a lower constant value in the domain of the foreground, and a higher constant value in the domain of the background to provide inverted density distribution form.
With the use of the plateau-like function, product sum calculation of pixel values of a pattern image and a corresponding input image upon calculation of the normalized correlation coefficient Cr can be reduced to summation of pixel values of the input image, achieving also smoothed and averaged noise of the input image. That is, implementation of pattern matching with noise smoothed contributes to a great improvement in the detection ability of pattern matching with high noise resistance.
Referring to
Referring to
Referring to
Then, an image of the selected regular pattern is overlaid on the original image of the object to confirm shape conformity therebetween (S3), and the foreground and background of the pattern image are separated from each other (S4). For the density of the foreground and background, there is a need to set an average density of each (S5, S6). The average density of the foreground is obtained, as a default, by averaging the density of the inside of the pattern image overlaid on the original image. The average density of the background is obtained by averaging the density of the outside of the pattern image. Optionally, the density of the foreground and background of the pattern image may be changed. Subsequently, an autocorrelation coefficient of the pattern image is calculated to use in calculation of the normalized correlation coefficient Cr (S7).
Referring to
Referring to
Then, calculation is made to obtain a cross-correlation coefficient between the pattern image and the input image (S27). The normalized correlation coefficient Cr is calculated by dividing the cross-correlation coefficient by the product of the autocorrelation coefficient of the pattern image and that of the input image.
Generally, the cross-correlation coefficient is obtained by product sum calculation of pixel values of the pattern image and the input image. In this case, because of being given by the rectangular-section function, the density distribution of the pattern image has constant values in the domains of the foreground and the background, only need is summation of pixel values of the input image.
A reduction of product sum calculation of pixel values of the pattern image and the input image to summation of pixel values of the input image contributes to not only smoothed noise as described above, but largely shortened calculation time. Particularly, a reduction in time for calculation of the cross-correlation coefficient which is repeatedly carried out in the process of scanning results in a large reduction in processing time for pattern matching.
When scanning is finished (S29), selection is made to obtain a maximum value of the normalized correlation coefficient Cr and a position of the pattern image (S30). It is noted that the position of the pattern image corresponds to a pattern detected position.
Generally, calculation of the normalized correlation coefficient Cr is depicted in “Digital Picture Processing” by Azriel Rosenfeld & Avinash C. Kak, pages 306–312, published by Kindai Kagaku Sha in 1992, the teachings of which are incorporated hereby by reference. The following is an example of a movement normalized correlation with no average values subtracted. The normalized correlation coefficient Cr is expressed by an equation (1):
Cr=∫Spf·tdS/(∫Spf2dS·∫Spt2dS)1/2 (1)
where S is a domain of definition of the integral, f is a density distribution function of an input image, and t is a density distribution function of the pattern image, and where ∫Spf·tds is a cross-correlation coefficient between the input image and the pattern image, ∫Spf2dS is an autocorrelation coefficient of the input image, and ∫Spt2dS is an autocorrelation coefficient of the pattern image and is constant.
As for a domain of definition Sp in the pattern image given in the equation (1), refer to
Using equations (2)–(4), the normalized correlation coefficient Cr can be expressed by an equation (5):
∥fz−t∥2=∫S(fz−t)2dS=(∫Sf2dS)z2−2(∫Sf−tdS)z+∫St2dS≧0 (2)
where z is a variable.
Discrimination function D=(f·tdS)2−∫f2dS·∫t2dS≦1 (3)
∴(∫f−tdS)2≦∫f2dS·∫t2dS (4)
where equality is obtained when fz−t=0.
Normalized correlation coefficient
Cr=∫f·tdS/(∫f2dS·∫t2dS)1/2≦1 (5)
where equality is obtained when t=fz (t f, i.e. t is similar to f).
As seen from the equation (5) when equality is obtained, if the density distribution function t of a pattern image is similar to that f of an input image, i.e. t f, the normalized correlation coefficient Cr is equal to 1 (maximum). It is thus desirable that the density distribution function t of a pattern image is similar to average values of density distribution of the foreground and background of an input image. Therefore, referring to
Thus, the cross-correlation coefficient between the input image and the pattern image in the equation (1) can be transformed into an equation (6):
∫Spf−tdS=∫SFf−tdS+∫SBf−tdS={overscore (t)}F∫SFfdS+{overscore (t)}B∫SBfdS (6)
where {overscore (t)}F and {overscore (t)}B are constant.
As seen from the equation (6), the cross-correlation coefficient discretized to product sum calculation becomes the integral of the density distribution function f of the input image discretized to summation, achieving smoothed noise, allowing pattern matching without being affected by noise.
Discretization of the equation (6) by pixels provides an equation (7):
As seen from the equation (7), product sum calculation of the density distribution functions f, t of the input image and the pattern image can be reduced to summation thereof, resulting in high-speed calculation.
Likewise, product sum calculation of the normalized correlation coefficient Cr with average values subtracted can be reduced to summation, obtaining the same effect.
As described above, an essential factor for increasing the detection ability of pattern matching comprises registration of pattern images. The shape of a pattern image as well as the density distribution of the foreground and background thereof is the key for increasing conformity of pattern matching between a pattern image and a corresponding input image.
According to the equation of definition of the normalized correlation coefficient Cr, achievement of high conformity (Cr=I) requires that the shape of the foreground of a pattern image is substantially equal to that of the foreground of a corresponding input image, and that the density distribution of the pattern image is similar to that of the input image.
The background of printed circuit boards may have various disturbances such as wiring layout pattern, silk-white circuit sign, partial dirt, resist, flux, etc., so that during pattern matching of regular lands, solders, mounted parts and lot numbers, non-matching occurs in the background of a pattern image and a corresponding input image. Thus, conformity is lowered as a whole to reduce a normalized correlation coefficient value, resulting in unsuccessful pattern matching.
In order to reduce a bad influence of disturbances, with the foreground and background of a pattern image being separated to remove the background, pattern matching is carried out, preferably, with regard to the foreground only. For this purpose, the density distributions of the foreground and background of the pattern image are separated at a stage of registration thereof.
In other illustrative embodiments of the present invention, recognition is carried out using the system as shown in
Referring to
In order to obtain smoothed noise and high-speed calculation of a correlation coefficient, an average density of an original image is designated with regard to the foreground and background thereof (S46, S46; S56, S57). Alternatively, due to background information excluded, the density may be designated such that the foreground is 1, and the background is 0. Optionally, it is possible to use the density distribution of an original image or the density distribution defined by a user.
Subsequently, an autocorrelation coefficient of the pattern image is calculated to use in calculation of the normalized correlation coefficient Cr (S47; S58). Since pattern matching is carried out with regard to only the foreground of the pattern image with the background excluded, an autocorrelation coefficient of the pattern image is calculated with regard to the foreground only, i.e. the foreground area of the pattern image (S47; S58).
This forms a mask for the background of the pattern image. Unlike a fixed mask for a particular portion of an input image, this mask is a kind of automatic mask which allows pattern matching with regard to only the foreground with the background masked during scanning of an input image enclosed in a search frame by a pattern image.
Referring to
Then, calculation is made to obtain a cross-correlation coefficient between the foreground area of the pattern image and the input image (S66). The normalized correlation coefficient Cr is calculated by dividing the cross-correlation coefficient by the product of the autocorrelation coefficient of the pattern image and that of the input image. Due to the background removed from the pattern image, pattern matching is ensured with bad influence of disturbances on the background minimized, resulting in shortened calculation time.
When scanning Is finished (S68), selection is made to obtain a maximum value of the normalized correlation coefficient Cr and a position of the pattern image (S69). It is noted that the position of the pattern image corresponds to a pattern detected position.
Referring to
The normalized correlation coefficient Cr is expressed by the above equation (1). Since the density distributions of the foreground and background of the pattern image have constant average values, respectively, the cross-correlation coefficient is expressed by an equation (8):
∫Spf−tdS={overscore (t)}F∫SFfdS+{overscore (t)}B∫SBfdS (8)
Foreground Background (Ignored)
Due to the background removed from a pattern image, a term of the background is ignored in the equation (8).
The autocorrelation coefficients are expressed by equations (9) and (10), respectively:
∫Spf2dS=∫SFf2dS+∫SBf2dS (9)
Foreground Background
(Variant)
∫Spt2dS={overscore (t)}F2sF+{overscore (t)}B2SB=const (10)
Foreground Background (Ignored)
A term of the background is ignored in the equation (10).
Thus, the normalized correlation coefficient Cr can be expressed by an equation (11) including only a term of the foreground with a term of the background excluded, allowing restrained bad influence of disturbances on the background and fast pattern-matching processing.
Cr={overscore (t)}F∫SFfdS/(∫SFf2dS−{overscore (t)}F2SF)1/2=∫SFf2dS−SF)1/2→max (11)
Likewise, the equation of the normalized correlation coefficient Cr with average values subtracted can be reduced to the equation including only a term of the foreground with a term of the background excluded, obtaining the same effect.
The inventive method allows real time pattern matching by means of processing in the CPU in a software way without using an exclusive fast image processing board, contributing to simplified apparatus structure and reduced manufacturing cost. Of course, the inventive method can be recorded as a computer program on readable recording media such as floppy disk and CD-ROM for various applications.
Having described the present invention in connection with the preferred embodiments, it is noted that the present invention is not limited thereto, and various changes and modifications can be made without departing from the scope of the present invention.
The entire contents of Japanese Patent Application P9-354322 are incorporated hereby by reference.
Number | Name | Date | Kind |
---|---|---|---|
5060282 | Molley | Oct 1991 | A |
5734747 | Vaidyanathan | Mar 1998 | A |
5748775 | Tsuchikawa et al. | May 1998 | A |
5757953 | Jang | May 1998 | A |
5790702 | Yoshimura | Aug 1998 | A |
5815597 | Horner et al. | Sep 1998 | A |
6115505 | Hashima et al. | Sep 2000 | A |
6240209 | Wilcke | May 2001 | B1 |
6330361 | Mitchell et al. | Dec 2001 | B1 |
6385340 | Wilson | May 2002 | B1 |
6389163 | Jodoin et al. | May 2002 | B1 |
Number | Date | Country |
---|---|---|
09-223233 | Aug 1977 | JP |