This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2019-012630, filed on Jan. 29, 2019, the entire contents of which are incorporated herein by reference.
The present disclosure relates to an image recognition system and an image recognition method.
In a manufacturing process of a semiconductor device, an electrical test is performed on a plurality of semiconductor devices formed on a semiconductor wafer (hereafter, briefly referred to as a wafer) after all processes for the wafer are finished. In apparatuses that perform such an electrical test, generally, a probe card having a plurality of probes that is configured to be brought into contact with a semiconductor device formed on a wafer is disposed to face a stage that adsorbs, holds and supports a wafer. Further, the wafer on the stage to the probe card is pressed to the probe card, whereby the probes of the probe card are brought into contact with an electrode pad of a device, and a test is performed on the electrical characteristic in this state.
In these test apparatuses, an image recognition technology that photographs an electrode pad and recognizes needle tracks from the image has been used to check whether probes have come into contact with the electrode pad of a device (e.g., Patent Document 1).
According to one embodiment of the present disclosure, there is provided an image recognition system including: an image data collector configured to collect image data including a recognition target from a plurality of test apparatuses; a learning processor configured to perform additional machine learning based on the image data collected in the image data collector for a first model, which is obtained by previous machine learning and configured to recognize a characteristic portion of the recognition target; a model updater configured to update a model configured to recognize the characteristic portion of the recognition target from the first model to a second model on a basis of a result of the additional machine learning by the learning processor; a first transmitter configured to transmit the second model to a specific test apparatus of the plurality of test apparatuses; a recognition result determiner configured to receive and determine a recognition result of recognizing the recognition target using the second model in the specific test apparatus; and a second transmitter configured to transmit the second model to at least one of the plurality of test apparatuses in accordance with a determination result by the recognition result determiner.
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure, and together with the general description given above and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.
Reference will now be made in detail to various embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one of ordinary skill in the art that the present disclosure may be practiced without these specific details. In other instances, well-known methods, procedures, systems, and components have not been described in detail so as not to unnecessarily obscure aspects of the various embodiments.
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.
First, a first embodiment is described.
The test system 400 includes a plurality of first test apparatuses 200 disposed in, for example, a factory, and an image recognition system 100 to improve a recognition level of recognition targets of the first test apparatuses 200 for image data including the recognition targets.
The first test apparatus 200, as shown in
The stage 201 can be aligned in a plane direction and an up-down direction by an aligner (not shown) and brings the probes 203 into contact with a plurality of test target devices formed on a wafer W, whereby an electrical test is performed by the tester 204. The first test apparatus 200 may perform a test by scanning a wafer W such that the probes 203 relatively scan the wafer W, and may perform a test by simultaneously bringing a plurality of probes into contact with a plurality of test target devices formed on a wafer W. The first test apparatus 200 may be a single test apparatus or may be a test apparatus having a plurality of test parts.
The first camera 205 is movable and is configured to photograph test target devices formed on a wafer W, as shown in
In the first test apparatus 200, for the recognition targets, the image recognizer 208 of the controller 207 is equipped with software that recognizes the recognition targets from image data using a model that is obtained in advance by machine learning to recognize characteristic portions of the recognition targets. The model can be updated by additional machine learning to be described below.
Machine learning means a technology and method that makes a computer, etc. perform the function of learning as humans naturally do. Deep learning may be appropriately used as the machine learning. Deep learning is a method of machine learning that uses a multi-layer neural network constructed by hierarchically connecting a plurality of processing layers. The model that is used in this case is a numerical formula, a plurality of parameters exists in the numerical formula, and the model can be changed by the values of the parameters or by giving weights. In this embodiment, the model in an initial state of the image recognizer 208 of the first test apparatus 200 is a first model #1.
The image recognition system 100, as shown in
The image data collector 10 collects image data including the recognition targets described above from the first test apparatuses 200. The image data that the image data collector 10 collects has only to be image data that could not be recognized by the image recognizers 208 of the first test apparatuses 200.
The learning processor 20 performs additional machine learning based on the image data collected in the image data collector 10 on a first model (the same as the first models #1 of the first test apparatuses 200) that is obtained by previous machine learning to recognize characteristic portions of the recognition targets. Deep learning described above is generally used as the machine learning in this case. Machine learning by the learning processor 20 may be automatically performed at an appropriate timing. Machine learning may be periodically performed or may be performed at a point in time when the data in the image data collector 10 reaches a predetermined amount. Machine learning may be performed by operation by an operator.
The model updater 30 updates the model for recognizing characteristic portions of the recognition targets from a first model #1 to a second model #2 on the basis of the result of the machine learning by the learning processor 20. The second model may be a model that can recognize image data from which the recognition targets could not be recognized in the first model, and then has a higher recognition level.
The learning processor 20 and the model updater 30 may be integrated.
The first transmitter 40 receives an updated second model from the model updater 30 and transmits the second model to a specific first test apparatus 200 of the plurality of first test apparatuses 200. The specific first test apparatus 200 performs recognition evaluation for the recognition targets of the image data from which the recognition targets could not be recognized in the first model, using the second model.
The recognition result determiner 50 receives and determines the recognition result of the recognition targets, of which recognition is performed using the second model in the specific first test apparatus 200, of image data which are similar to or same as image data from which the recognition targets could not be recognized in the first model.
The second transmitter 60 transmits the second model to first test apparatuses 200 other than the specific first test apparatus in accordance with the determination result by the recognition result determiner 50. In more detail, the second transmitter 60 transmits the second model to the first test apparatuses 200 when the recognition result determiner 50 determines that the recognition result is satisfactory.
Next, an image recognition method in the image recognition system 100 according to the first embodiment is described.
First, image data including the recognition targets are collected into the image data collector 10 from a plurality of test apparatuses 200 installed in a factory (ST1). The image data may be ones that could not be recognized by the image recognizers 208 of the first test apparatuses 200.
Next, the learning processor 20 performs additional machine learning on a first model, which is obtained in previous machine learning to recognize characteristic portions of the recognition targets, on the basis of the image data collected in the image data collector 10 (ST2). Deep learning described above is generally used as the machine learning in this case.
Next, the model updater 30 updates the model for recognizing characteristic portions of the recognition targets from the first model to a second model on the basis of the result of the machine learning (ST3).
Next, the second model is transmitted to a specific first test apparatus of the plurality of first test apparatuses 200 installed in the factory (ST4).
Next, recognition evaluation is performed for the recognition targets using the second model in the specific first test apparatus 200 (ST5).
Next, the recognition result performed in the specific first test apparatus is determined (ST6).
Next, the second model is transmitted from the second transmitter 60 to the first test apparatuses 200 on the basis of the determination result in ST6 (ST7). In detail, when it is determined that the recognition result is satisfactory by the recognition result determiner 50 in ST6, the models of the image recognizers of all the test apparatuses 200 are updated to second models by transmitting the second model to all the first test apparatuses 200 in the factory.
If the recognition result in the specific first test apparatus 200 is not satisfactory, updating to the second model is stopped. Although only recognition evaluation for the second model in the specific first test apparatus 200 is performed in ST5 in the above embodiment, the model of the first test apparatus 200 may be updated to the second model in ST5. In this case, when it is determined that the recognition result is satisfactory by the recognition result determiner 50 in ST6, the second model is transmitted to the first test apparatuses 200 other than the specific first test apparatus 200 in ST7.
As described above, according to the image recognition system 100 of this embodiment, image data including the recognition targets are collected into the image data collector 10 from a plurality of first test apparatuses 200 installed in the factory. For the first model, additional machine learning is performed on the basis of the collected image data and the model is updated from the first model to the second model. After the recognition result of the second model in the specific first test apparatus 200 is recognized, the models of all the first test apparatuses 200 in the factory are replaced with the second model. Accordingly, it is possible to always recognize the recognition targets on the basis of a newly updated model in all the first test apparatuses 200 in the factory. Therefore, even if image recognition is not accurately performed on a recognition target, it is possible to efficiently recognize the recognition target within a short time. Further, it is possible to recognize the recognition target without leaking information out of the factory.
In the related art, as described in Patent Document 1, the recognition targets such as a needle track were photographed by a camera and recognized as images. However, updating software for performing image recognition on the recognition targets is not described in Patent Document 1.
In general, in such type of image recognition for the recognition targets, for example, when a new device is tested or when a recognition target is changed by a change according to a lapse of time, whether there is dust, a difference in contrast, etc., it is impossible to deal with this situation with the existing software, so a faulty recognition result in which image recognition is not accurately performed occurs.
For example, when a recognition target is a needle track, if there is dust 503 other than the needle track 502, as shown in
In the related art, when this situation occurs and a faulty recognition result is obtained, people involved with the fields of service and technique need to do even examination of a reform measure, and design, construction, evaluation, and installation of software including collection of images. Further, it takes long time to operate improved software after a faulty recognition result is obtained.
However, in the image recognition system 100 according to this embodiment, as described above, even if image recognition is not accurately performed on a recognition target, it is possible to efficiently recognize the recognition target within a short period by machine learning. Further, since it is possible to recognize a recognition target without leaking information out of the factory, it is possible to obtain an effect of preventing information leakage. Further, it is possible to recognize the recognition targets by the same model for all first test apparatuses 200 in the factory and it is possible to recognize the recognition targets at the same level in the factory.
Further, as a modified example of the image recognition system 100, as shown in
Next, a second embodiment is described.
The test system 401 includes the plurality of first test apparatuses 200 disposed in, for example, a factory, one or more second test apparatuses 300, and an image recognition system 101 for improving a recognition level of the recognition targets of the first test apparatuses 200 and the second test apparatuses 300 from image data including the recognition targets.
The second test apparatuses 300 are the same in fundamental configuration as the first test apparatuses 200), but are different from the first test apparatuses 200 in that the software of the image recognizer does not use a model obtained by machine learning to recognize characteristic portions of the recognition targets.
The image recognition system 101 according to this embodiment, as shown in
The learning processor 20, the model updater 30, the first transmitter 40, the recognition result determiner 50, and the second transmitter 60 are configured in the same way as the first embodiment.
The estimation image data collector 110 collects image data including the recognition targets from the first test apparatuses 300. The image data that the estimation image data collector 110 collects has only to be image data that could not be recognized by the image recognizers of the second test apparatuses 300.
The estimator 120 receives image data including the recognition targets from the estimation image data collector 110 and estimates the recognition targets for the image data. In detail, the estimator 120 estimates the recognition targets using the first model described above.
The software of the image recognizer does not use a model for recognizing characteristic portions of the recognition targets, so the second test apparatuses 300 estimates the recognition targets using a first model by the estimator 120, similar to the first test apparatuses 200. The estimator 120 can receive information from the model updater 30 and update the first model to a second model. Accordingly, the estimator 120 can estimate the recognition targets using the second model.
The data processor 130 transmits the result of estimating (recognizing) the recognition targets by the estimator 120 to the second test apparatus 300 of a transmission source of image data. When the estimator 120 fails to estimate (recognize) a recognition target, the data processor 130 transmits the estimation result of the recognition target to the second test apparatus 300 of a transmission source of image data and accumulates the image data in the image data collector 10. The estimation result that is transmitted to the second test apparatus 300 is numerical data (the position, size, etc. of a needle track when a recognition target is a needle track). When the estimator 120 can estimate (recognize) the recognition targets, the image data is deleted from the estimation image data collector 110.
Next, an image recognition method in the image recognition system 101 according to the second embodiment is described.
Image data are collected into the estimation image data collector 110 from the second test apparatuses 300 installed in the factory (ST11). The image data that are collected in this case has only to be image data that could not be recognized by the image recognizers of the second test apparatuses 300.
The estimator 120 estimates the recognition targets for the image data collected by the estimation image data collector 110 (ST12). In this process, it is possible to estimate the recognition targets using the first model described above. Accordingly, recognition of the recognition targets of the second test apparatuses 300 can be performed with the same level as that of the first test apparatuses 200. When the model updater 30 has updated the model from a first model to a second model, ST12 can be performed by receiving information from the model updater 30 and updating the first model to the second model.
When the recognition target can be estimated in ST12, the result is transmitted to the second test apparatus 300 of a transmission source of image data. When the recognition target cannot be estimated, the result is transmitted to the second test apparatus 300 of a transmission source of image data and the image data is accumulated in the image data collector 10 (ST13). Accordingly, it is possible to collect image data for model update even from the second test apparatus 300. The factor that is transmitted to the second test apparatus 300 in ST13 is numerical data (the position, size, etc. of a needle track when a recognition target is a needle track). When the recognition target can be estimated, the image data is deleted from the estimation image data collector 110.
In this embodiment, other than these processes, ST1 to ST7 of the first embodiment are performed. ST11 to ST13 are described over ST1 to ST7 in
According to the image recognition system 101 of this embodiment, similar to the image recognition system 100 of the first embodiment, the models of all the first test apparatuses 200 are replaced with second models by collection of image data including the recognition targets in the image data collector 10, additional machine learning, and model update. Accordingly, it is possible to always recognize the recognition targets on the basis of a newly updated model in all the first test apparatuses 200 in the factory. Therefore, even if image recognition is not accurately performed on a recognition target, it is possible to efficiently recognize the recognition target within a short time without leaking information out of the factory. Further, even if there is a second test apparatus 300 that does not use a model obtained by machine learning in the factory, it is possible to make a recognition level for the recognition targets be a level close to the case of only the first test apparatuses 200. Further, in the second test apparatuses 300, even the image data including the recognition targets, which could not be estimated by the estimator 120, are collected in the image data collector 10 and used as image data for additional machine learning, thereby being able to contribute to upgrading a model.
Further, as a modified example of the image recognition system 101, as shown in
Next, a third embodiment is described.
An image recognition system 102 according to this embodiment, as shown in
In the image recognition system 102 according to this embodiment, similar to the image recognition system 101 according to the second embodiment, image data are collected into the estimation image data collector 110 from the second test apparatuses 300 installed in the factory. The image data that are collected in this case has only to be image data that could not be recognized by the image recognizers of the second test apparatuses 300. The estimator 120 estimates the recognition targets for the image data collected by the estimation image data collector 110 using the first model described above. The data processor 130 transmits the estimation result for the recognition targets to the second test apparatus 300 that is a transmission source of image data. When the estimator 120 fails to estimate (recognize) a recognition target, the data processor 130 transmits the estimation result to the second test apparatus 300 of a source of image data and accumulates the image data in the image data collector 10. Image data from which a recognition target could not be recognized are collected in the image data collector 10 even from the first test apparatuses 200.
Accordingly, as test apparatuses in the factory, when there is a first test apparatus 200 that uses machine learning using a first model and a second test apparatus 300 that does not use machine learning, the second test apparatus 300 can also recognize the recognition targets with the same level as that of the first test apparatus 200. Further, image data from which a recognition target could not be recognized by the first model can be accumulated in the image data accumulator 10 from both of the first test apparatus 200 and the second test apparatus 300. Accordingly, the image data are provided to a separately provided machine learning processor, whereby it is possible to update a model and improve the recognition level for the recognition targets.
Further, an image recognition system 103 shown in
Although embodiments were described above, the embodiments disclosed herein should be construed as only examples, not limiting, in all terms. The above embodiments may be omitted, replaced, and changed in various ways without departing from the accompanying claims and the subject thereof.
For example, the test apparatuses of the embodiments are only examples and any test apparatus can be applied as long as it includes an operation that recognizes the recognition target by image recognition.
Further, although an electrode pad, needle tracks formed on an electrode pad by probes, and needle tips of probes were exemplified as the recognition targets in the embodiments described above, the recognition targets are not limited thereto.
According to the present disclosure, there are provided an image recognition system and an image recognition method that can recognize the recognition targets within a short period even if image recognition is not accurately performed on the recognition target such as a needle track in test apparatuses in, for example, a factory.
While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the disclosures. Indeed, the embodiments described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the disclosures. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the disclosures.
Number | Date | Country | Kind |
---|---|---|---|
2019-012630 | Jan 2019 | JP | national |