Patent Application
20240096081
The present invention relates to a method, apparatus and computer program for training a neural network
When measuring samples using a scanning probe microscope, it is often important to accurately measure features of these samples, such as the height of a protrusion or the depth of a depression. To measure height for example, it is necessary to measure a point of the feature relative to a “ground” level of the sample. To do this efficiently, and accurately on an automated basis, it is important to be able to reliably identify the feature, as well as suitable “ground” points in the image (i.e. those positions not relating to a feature or abnormality). This can be done using a neural network. There is therefore a need to train the neural network to be able to perform this function.
A first aspect of the invention provides a method of training a neural network for use in surface metrology, comprising: providing height image data comprising a series of height measurements of a sample, the height image data comprising a plurality of features; obtaining, from the height image data, a plurality of height image patches, each height image patch containing at least a portion of a feature; applying one or more effects to each of the height image patches to obtain a corresponding modified height image patch for each height image patch; inputting one or more of the modified height image patches into the neural network; using the neural network to identify a feature in each of the one or more modified height image patches; and training the neural network based on the identification.
The height image data may comprise real data obtained from a real sample. For example the height image data may comprise real data obtained from a real sample by scanning the real sample with a probe microscope. Additionally or alternatively, the height image data may comprise simulated data.
The one or more effects may comprise at least one of: a rotation; a reflection; applying noise; raising or lowering brightness; raising or lowering contrast; zooming in; and zooming out.
The plurality of height image patches may comprise 10000 height image patches. The plurality of height image patches may comprise 15000 height image patches.
The neural network may be trained based on each of the modified height image patches.
Each height image patch may comprise at least a corner, edge or central portion of a feature.
The method may further comprise: obtaining, from the height image data, a plurality of additional height image patches that do not contain a feature or a portion of a feature; applying one or more effects to each of the additional height image patches to obtain a corresponding modified additional height image patch for each additional height image patch; inputting one or more of the modified additional height image patches into the neural network; using the neural network to determine that there are no features in each of the one or more modified additional height image patches; and training the neural network based on the determination.
Optionally each height image patch comprises height measurements of an area of a surface of the sample.
Optionally the method further comprises: obtaining, from the height image data, height image patches which do not contain a portion of a feature.
Optionally a plurality of the features have at least a portion of the feature contained in at least one of the height image patches.
Optionally some of the height image patches contain an entire feature and some of the height image patches contain a portion of a feature.
A second aspect of the invention provides apparatus for training a neural network for use in surface metrology, comprising a processor configured to perform the method of the first aspect.
A third aspect of the invention provides a method of performing surface metrology of a sample, the method comprising: training a neural network by a method according to the first aspect, thereby generating a trained neural network; scanning a new sample to obtain a series of height measurements of the new sample; and operating the trained neural network to identify feature data in the series of height measurements of the new sample.
A fourth aspect of the invention provides a computer-readable medium that, when read by a computer, causes the computer to perform the method of the first aspect.
Embodiments of the invention will now be described with reference to the accompanying drawings, in which:
A scanning probe microscopy system according to an embodiment of the invention is shown in
The probe tip 3 comprises a conical or pyramidal structure that tapers from its base to a point at its distal end that is its closest point of interaction with a sample 7 on a sample stage 11a. The sample comprises a sample surface which defines a sample surface axis which is normal to the sample surface and in
In this example the probe tip 3 tapers to a point, but in other embodiments the probe tip 3 may be specially adapted for measuring sidewalls. For instance the probe tip 3 may have a flared shape.
The cantilever 2 is a thermal bimorph structure composed of two (or more) materials, with differing thermal expansion coefficients—typically a silicon or silicon nitride base with a gold or aluminium coating. The coating extends the length of the cantilever and covers the reverse side from the tip 3. An illumination system (in the form of a laser 30) under the control of photothermal (PT) drive 33 is arranged to illuminate the cantilever on its upper coated side with an intensity-modulated radiation spot.
The cantilever 2 is formed from a monolithic structure with uniform thickness. For example the monolithic structure may be formed by selectively etching a thin film of SiO2 or SiN4 as described in Albrecht T., Akamine, S., Carver, T. E., Quate, C. F. J., Microfabrication of cantilever styli for the atomic force microscope, Vac. Sci. Technol. A 1990, 8, 3386 (hereinafter referred to as “Albrecht et al.”). The tip 3 may be formed integrally with the cantilever, as described in Albrecht et al., it may be formed by an additive process such as electron beam deposition, or it may be formed separately and attached by adhesive or some other attachment method.
The wavelength of the actuation beam 32 output by the laser 30 is selected for good absorption by the coating, so that the cantilever 2 bends along its length and moves the probe tip 3. In this example the coating is on the reverse side from the sample so the cantilever 2 bends down towards the sample when heated, but alternatively the coating may be on the same side as the sample so the cantilever 2 bends away from the sample when heated.
The piezoelectric driver 4 expands and contracts up and down in the Z-direction in accordance with a piezo drive signal 5 at a piezo driver input. As described further below, the piezo drive signal 5 causes the piezoelectric driver 4 to move the probe repeatedly towards and away from the sample 7 in a series of cycles. The piezo drive signal 5 is generated by a piezo controller (not shown). Typically the piezoelectric driver 4 is mechanically guided by flexures (not shown).
A measurement system 80 is arranged to detect a height of the free end of the cantilever 2 directly opposite to the probe tip 3. The measurement system 80 includes an interferometer and a quadrant photodiode (QPD).
Ideally, the outputs from the photodetectors 121, 122 are complementary sine and cosine signals with a phase difference of 90 degrees. Further, they should have no dc offset, have equal amplitudes and only depend on the position of the cantilever and wavelength of the laser 101. Known methods are used to monitor the outputs of the photodetectors 121, 122 while changing the optical path difference in order to determine and to apply corrections for errors arising as a result of the two photodetector outputs not being perfectly harmonic, with equal amplitude and in phase quadrature. Similarly, dc offset levels are also corrected in accordance with methods known in the art.
These photodetector outputs are suitable for use with a conventional interferometer reversible fringe counting apparatus and fringe subdividing apparatus 123, which may be provided as dedicated hardware, FPGA, DSP or as a programmed computer. Phase quadrature fringe counting apparatus is capable of measuring displacements in the position of the cantilever to an accuracy of λ/8. That is, to 66 nm for 532 nm light.
Known fringe subdividing techniques, based on the arc tangent of the signals, permit an improvement in accuracy to the nanometre scale or less. In the embodiment described above, the reference beam 104 is arranged to have a fixed optical path length relative to the Z position of the sample 7. It could accordingly be reflected from the surface of the stage 11a on which the sample 7 is mounted or from a retro-reflector whose position is linked to that of the stage. The reference path length may be greater than or smaller than the length of the path followed by the beam 103 reflected from the probe. Alternatively, the relationship between reflector and sample Z position does not have to be fixed. In such an embodiment the reference beam may be reflected from a fixed point, the fixed point having a known (but varying) relationship with the Z position of the sample. The height of the tip is therefore deduced from the interferometrically measured path difference and the Z position of the sample with respect to the fixed point.
The interferometer detector is one example of a homodyne system. The particular system described offers a number of advantages to this application. The use of two phase quadrature interferograms enables the measurement of cantilever displacement over multiple fringes, and hence over a large displacement range. Examples of an interferometer based on these principles are described in U.S. Pat. No. 6,678,056 and WO2010/067129. Alternative interferometer systems capable of measuring a change in optical path length may also be employed. A suitable homodyne polarisation interferometer is described in EP 1 892 727 and a suitable heterodyne interferometer is described in U.S. Pat. No. 5,144,150.
Returning to
The reflected beam is also split by a beam splitter 106 into first and second components 107, 110. The first component 107 is directed to a segmented quadrant photodiode 108 via a lens 109, and the second component 110 is split by the beam splitter 102 and directed to the photodiodes 121, 122 for generation of the height signal on the output line 20. The photodiode 108 generates angle data 124 which is indicative of the position of the first component 107 of the reflected beam on the photodiode 108, and varies in accordance with the angle of inclination of the cantilever relative to the sensing beam 103.
The angle data 124 comprises a deflection/bending signal which indicates a flexural angle of the cantilever—i.e. an angle which changes as the cantilever bends along its length. Thus the deflection/bending signal is indicative of the flexural shape of the cantilever. The deflection/bending signal may be determined in accordance with a difference between the signals from the top and bottom halves of the quadrant photodiode 108.
The angle data 124 also comprises a lateral/twisting signal which indicates a torsion angle of the cantilever—i.e. an angle which changes as the cantilever twists. Thus the lateral/twisting signal is indicative of the torsional shape of the cantilever. The lateral/twisting signal may be determined in accordance with a difference between the signals from the left and right halves of the quadrant photodiode 108.
The scanning probe microscopy system described above is used to scan a sample to obtain probe microscope data in the form of a series of height image data. The sample comprises a plurality of features. A feature in the sample is simply an abnormality of the sample, optionally having a height dimension that notably deviates from a mean or mode height dimension across the sample. The features may be columns or wells, for example. Features correspond to feature data within height image data.
Turning to
The height image data 201 is divided into a plurality of height image patches 202. Each height image patch comprises height measurements of an area of the surface of the sample, each height measurement being represented by a respective pixel.
In
A plurality of the features may have at least a portion of the feature contained in at least one of the height image patches. In the case of
Some of the height image patches 202 in
One or more effects are applied to each of the height image patches to obtain a corresponding modified height image patch 203 for each height image patch, as can be seen in
Each modified height image patch is input into the neural network, and the neural network is used to identify any features or portions of features in the modified height image patch. The patches in which features have been identified (“identified patches” 303) are shown alongside their respective modified height image patches. In the identified patches 303, the feature data appears as white, while the non-feature data appears as black. For each identified patch 303, the neural network is scored based on how accurately the features are identified. The neural network may also be provided with the “answer”, i.e. the correct feature identification—the correct features are manually labelled if real data is being used, or are already known and automatically output if simulated data is being used (discussed further below). The neural network is trained using this scoring. For example, if the neural network scores well for a particular patch, it will “learn” that its identification for that patch was good, and that it should reinforce the identification patterns used for that patch. Conversely, if the neural network scores poorly for a given patch, its algorithm/code may be altered so as to more accurately identify features in the future. This alteration may be based on the correct feature identification.
Alternatively or additionally, the neural network may simply be provided with both the modified height image patches and the corresponding identified patches at the same time, in order to learn to correctly identify the features without a scoring system.
Although this embodiment has been described using real height image data that was obtained from a scan performed by a scanning probe microscope, it will be appreciated that the neural network could equally or additionally be trained using height image data that has been simulated by a computer. For example, the height image patches could be obtained using data generated by a computer simulation. In other words, simulated data is created using a computer model of the physical probe interaction against samples. Samples and structures are defined in code, and the scanning probe microscope interactions are simulated on the virtual sample. Alternatively, the height image patches could be a combination of real height image data and simulated height image data.
Turning to
Once trained, the neural network segments the height image data so as to identify the feature data. In other words, the trained neural network receives the height image data as an input, and determines the portions of the image that correspond to the feature data 403 and the portions of the image that do not correspond to the feature data 403. These determinations mean that the neural network can identify the feature data 403.
The neural network can output a mask, using the identified feature data 403a. As shown in
The transformation may be a first order transformation, for example, in which an angled plane is transformed to a horizontal plane. Alternatively, the transformation may be a second order transformation or higher order transformation, in which a curved surface is transformed to a horizontal plane.
As with the previous embodiment, the trained neural network segments the height image data so as to identify the feature data. The result of this identification 503a is shown in
Turning now to
Specifically, though not shown in this Figure, the images 602a, 602b are provided to the neural network as modified height image patches 202, and the correctly identified images 603a, 603b may be provided at the same time as the modified height image patches 202, or afterwards, for scoring purposes or alongside a scoring result.
Following the training of the neural network to generate a trained neural network, a new sample is scanned with a probe microscope to obtain a series of height measurements of the new sample, as represented by a new image 604. The trained neural network is then operated to identify feature data in the series of height measurements of the new sample. For example the trained neural network 601 may segment the new image 604 to identify any feature data in the manner described above. This identification may be used to generate and output a mask 605.
In contrast, the segmentation 703 performed by the trained neural network using the method described above is shown in
Turning now to
Although the invention has been described above with reference to one or more preferred embodiments, it will be appreciated that various changes or modifications may be made without departing from the scope of the invention as defined in the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2213576.8 | Sep 2022 | GB | national |
