This U.S. non-provisional patent application is based on and claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2021-0149022, filed on Nov. 2, 2021, in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety.
The inventive concept(s) described herein relate to three-dimensional modeling, and more particularly, to a method and system of modeling a three-dimensional structure.
A simulation based on a three-dimensional structure may incur high cost. For example, in simulating an attribute of a device formed under a condition in a semiconductor process or simulating a state of the device in a predetermined environment, a high computing resource for performing various physical interpretations may be needed, and a long time may be taken in completing a simulation. Also, due to performance or various factors of a simulator, the accuracy of a simulation result may decrease.
The inventive concept(s) described herein provide a method and system for modeling a three-dimensional structure at low cost and high accuracy.
According to an aspect of the present disclosure, a 3D modeling (three-dimensional modeling) method includes obtaining geometric data representing a 3D structure and input parameters including factors determining an attribute of the 3D structure, generating grid data from the geometric data, sequentially generating at least one piece of down-sampled data from the grid data, generate a 3D feature map by pre-processing the input parameters, and generating attribute profile data, representing a profile of the attribute in the 3D structure, from the at least one piece of down-sampled grid data and the 3D feature map based on at least one machine learning model respectively corresponding to at least one stage.
According to another aspect of the present disclosure, a 3D modeling (three-dimensional modeling) method including obtaining attribute profile data representing a profile of an attribute of a 3D structure and input parameters representing an environment of the 3D structure, generating grid data from the geometric data, sequentially generating at least one piece of down-sampled data from the grid data, generating a 3D feature map from the grid data, the at least one piece of down-sampled grid data, and the input parameters based on at least one first machine learning model respectively corresponding to at least one first stage, and generating state data representing a state of the 3D structure in the environment by post-processing the 3D feature map.
According to another aspect of the present disclosure, there is provided a system including at least one processor and a non-transitory storage medium configured to store instructions allowing the at least one processor to perform a 3D modeling method when the instructions are executed by the at least one processor.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium including instructions allowing the at least one processor to perform a 3D modeling method when the instructions are executed by the at least one processor.
Embodiments of the present disclosure will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings in which:
In some embodiments, the 3D modeling 10 of
The geometric data D11 may represent a 3D structure for modeling or simulation. For example, the geometric data D11 may be data representing a 3D structure for simulating an attribute of a 3D structure by using a simulator. As illustrated in
The interpolation 12 may generate grid data D12 from the geometric data D11. In order to be learned or inferred by a machine learning model (e.g., by at least one up-sampling based model 18 described below), the grid data D12 provided to the at least one up-sampling based model 18 may be based on a uniform grid. A uniform grid may have a constant interval between elements (i.e., may have the same interval between adjacent intersections in and throughout one or more dimensions). As described above, the geometric data D11 may be based on a non-uniform grid, and thus, a grid of the geometric data D11 may be interpolated, whereby the grid data D12 having a uniform grid may be generated. In some embodiments, an interval in a grid of the grid data D12 may correspond to a minimum interval in a grid of the geometric data D11. Also, in some embodiments, the grid data D12 may be based on a Cartesian coordinate system and may include values respectively corresponding to X, Y, and Z directions. Examples of the grid data D12 will be described below with reference to
In the down-sampling 14, the grid data D12 may be down-sampled. As described below, a 3D feature map D16 generated from the input parameters D15 may have a low resolution. Therefore, pieces of down-sampled grid data D13 and D14 generated from the grid data D12 may be provided to the at least one up-sampling based model 18. In some embodiments, the grid data D12 may be provided to a network (for example, a convolution neural network) to generate the pieces of down-sampled grid data D13 and D14. Also, in some embodiments, the down-sampling 14 may be based on max pooling and average pooling.
The input parameters D15 may include factors for determining an attribute of a 3D structure. For example, the input parameters D15 may include process parameters for determining a doping profile of a semiconductor device, and for example, may include a dopant, a dose, an implantation tilt, implantation energy, and a temperature. In some embodiments, the input parameters D15 may be the same as parameters provided to a simulator.
The pre-processing 16 may generate a 3D feature map from the input parameters D15. The input parameters D15 may be a series of values and may correspond to one-dimensional (1D) data. The 3D feature map D16 may be generated from the input parameters D15 and is provided to an up-sampling based model along with the grid data D12 including geometric information about a 3D structure and the down-sampled grid data D13 and D14 generated from the grid data D12. An example of the pre-processing 16 will be described below with reference to
The at least one up-sampling based model 18 may receive the grid data D12, the down-sampled grid data D13 and D14, and the 3D feature map D16 and may output attribute profile data D17. Each of the at least one up-sampling based model 18 may be a machine learning model and may be trained to output grid data and a 3D feature map. As described below with reference to
For example, at least one piece of down-sampled data from the down-sampling 14 may be sequentially generated by generating second grid data (e.g., the down-sampled grid data D13 and D14) via down-sampling first grid data (e.g., the grid data D12). A first machine learning model (e.g., one of the at least one up-sampling based model 18) may be executed based on output data of a previous stage, the first grid data, and the second grid data. The first machine learning model may be executed to generate the attribute profile data D17. As explained below, the executing of the first machine learning model may include concatenating the output data of a previous first stage with the first grid data, executing a convolution layer based on concatenated data, down-sampling output data of the convolution layer, and executing a series of residual blocks based on down-sampled data and the second grid data.
Herein, a machine learning model may have an arbitrary trainable structure. For example, the machine learning model may include an artificial neural network, a decision tree, a support vector machine, a Bayesian network, and/or a genetic algorithm. Hereinafter, the machine learning model will be described with reference to an artificial neural network, but embodiments are not limited thereto. The artificial neural network, as a non-limiting example, may include a convolution neural network (CNN), a region with convolution neural network (R-CNN), a region proposal network (RPN), a recurrent neural network (RNN), a stacking-based deep neural network (S-DNN), a state-space dynamic neural network (S-SDNN), a deconvolution network, a deep belief network (DBN), a restricted Boltzmann machine (RBM), a fully convolutional network, a long short-term memory (LSTM) network, and a classification network. Herein, the machine learning model may be simply referred to as a model.
According to an experiment result, the 3D modeling 10 of
Each of the grid data 20a of
The 3D feature map D16 generated by the pre-processing 16 described above with reference to
In some embodiments, an attribute of a portion of a 3D structure (e.g., less than all of the 3D structure) may be considered. For example, as illustrated in
In some embodiments, the pre-processing 30 may include and/or involve a network and may include at least one layer. For example, as illustrated in
Referring to
An up-sampling based model (for example, 41) in each stage may output a 3D feature map having a higher resolution than that of an input 3D feature map (for example, D41) based on grid data D42 having a relatively lower resolution and grid data D43 having a relatively higher resolution. Therefore, attribute profile data D46 representing an attribute profile of a 3D structure may be generated from a 3D feature map D41 generated by pre-processing the input parameters D15 of
Hereinafter, the 3D feature map Xi having a lower resolution may be referred to as an input feature map Xi, and the 3D feature map Xi+1 having a higher resolution may be referred to as an output feature map Xi+1. Also, the grid data D51 having a higher resolution may be referred to as first grid data D51, and the grid data D52 having a lower resolution may be referred to as second grid data D52. The input feature map Xi may be a 3D feature map (for example, D41 of
Referring to
A result of the concatenation 51 may be provided to the convolution 53 (or a convolution layer), and a result of the convolution 53 may be provided to the up-sampling 55 (or an up-sampling layer). The up-sampling 55 may be performed based on an arbitrary scheme, and for example, a value may be copied to added grids. Therefore, a result of the up-sampling 55 may have a resolution which is higher than that of each of the input feature map Xi and the second grid data D52. The result of the up-sampling 55 may be provided to the series of residual blocks 57 and 59. High complexity caused by a deep network in deep learning may require a number of resources, and a depth of a network may not be proportional to performance of the network. In order to solve such a problem, residual learning may be used. The residual learning may denote that low-resolution data is added to high-resolution data and a difference value between two pieces of data is learned. For example, in ResNet proposed in the paper “Deep Residual Learning for Image Recognition”, a network may be divided into a plurality of residual blocks so as to more stably train a deep network and the plurality of residual blocks may be connected to one another through a skip connection, and thus, filter parameters may be more easily optimized. As illustrated in
As an example implementation of embodiments based on
Referring to
In the first concatenation 61, values of the input Yj may be concatenated with values of the grid data D60, and a result of the first concatenation 61 may be provided to the first convolution 62. In some embodiments, a filter in the first convolution 62 may have a 1×1 size, and the first convolution 62 may include filters corresponding to ¼ of the number of filters included in the third convolution 66. In some embodiments, as illustrated in
In the second concatenation 63, a result of the first convolution 62 may be concatenated with the values of the grid data D60, and a result of the second concatenation 63 may be provided to the second convolution 64. In some embodiments, a filter in the second convolution 64 may have a 3×3 size, and the second convolution 64 may include filters corresponding to ¼ of the number of filters included in the third convolution 66. In some embodiments, as illustrated in
In the third concatenation 65, a result of the second convolution 64 may be concatenated with the values of the grid data D60, and a result of the third concatenation 65 may be provided to the third convolution 66. In some embodiments, the third convolution 66 may include a plurality of filters, and each of the plurality of filters may have a 1×1 size. In some embodiments, as illustrated in
The adder 67 may add the input Yj to an output of the third convolution 66. In some embodiments, as illustrated in
As described above with reference to
In some embodiments, the 3D modeling 80 of
The attribute profile data D81 may three-dimensionally represent an attribute of a 3D structure. For example, the attribute profile data D81 may be generated by the 3D modeling 10 and the interpolation 70 of
The interpolation 81 may generate grid data D82 from the attribute profile data D81. The grid data D82 provided to the at least one down-sampling based model 85 may be based on a uniform grid, so as to be learned or inferred by the at least one down-sampling based model 85 described below. As described above, the attribute profile data D81 may be based on a non-uniform grid. Therefore, a grid of the attribute profile data D81 may be interpolated, whereby the grid data D82 having a uniform grid may be generated. Unlike the grid data D12 of
In the down-sampling 83, the grid data D82 may be down-sampled. As described below, a 3D feature map D86 generated in the at least one down-sampling based model 85 may have a low resolution. Therefore, pieces of grid data D83 and D84 down-sampled from the grid data D82 may be provided to the at least one down-sampling based model 85. In some embodiments, the grid data D82 may pass through a network (for example, a CNN) to generate the down-sampled grid data D83 and D84. Also, in some embodiments, the down-sampling 83 may be performed based on max pooling, average pooling, or the like.
The input parameters D85 may include factors for determining a state of a 3D structure. For example, the input parameters D85 may include parameters representing an environment of a 3D structure such as a voltage and a temperature each provided to a semiconductor device. In some embodiments, the input parameters D85 may be the same as parameters provided to a simulator. When the attribute profile data D81 represents a doping profile of a transistor and the input parameters D85 represent voltages applied to the transistor, state data D87 may represent a current characteristic (for example, a voltage-current graph), and state profile data D88 may represent a density of an electron and/or a hole in the transistor.
The at least one down-sampling based model 85 may receive the grid data D82, the down-sampled grid data D83 and D84, and the input parameters D85 and may output the 3D feature map D16. Each of the at least one down-sampling based model 85 may be a machine learning model and may be trained to output grid data and a 3D feature map. As described above with reference to
Based on the post-processing 87, the state data D87 may be generated from the 3D feature map D86. The state data D86 may represent a state of a 3D structure having an attribute of the attribute profile data D81 under a condition corresponding to the input parameters D85. An example of the post-processing 87 will be described below with reference to
The at least one up-sampling based model 89 may output the state profile data D88 from the 3D feature map D86. For example, the at least one up-sampling based model 89 may have the same structure as that of the at least one up-sampling based model 40 of
Compared to a simulator, the 3D modeling 10 of
Referring to
A down-sampling based model (for example, the first down-sampling based model 91) in each stage may output a 3D feature map having a higher resolution than that of input data (for example, D91) based on grid data D92 having a higher resolution and grid data D93 having a lower resolution. Therefore, a 3D feature map D96 representing a state of a 3D structure from the input parameters D91 may be generated by a series of down-sampling based models. As described below with reference to
Hereinafter, the 3D feature map Zk having a higher resolution may be referred to as an input feature map Zk, and the 3D feature map Zk+1 having a lower resolution may be referred to as an output feature map Zk+1. Also, the grid data D101 having a higher resolution may be referred to as first grid data D101, and the grid data D102 having a lower resolution may be referred to as second grid data D102. Referring to
In the concatenation 101, values of the input feature map Zk may be concatenated with values of the first grid data D101. When the down-sampling based model 100 is included in the first stage, the input feature map Zk may be 1D input parameters D91, and values of the input parameters D91 may be respectively concatenated with values of the first grid data D101.
A result of the concatenation 101 may be provided to the convolution 103 (or a convolution layer), and a result of the convolution 103 may be provided to the down-sampling 105 (or a down-sampling layer). The down-sampling 105 may be performed based on an arbitrary scheme, and for example, may be based on max pooling, average pooling, and a (convolution) neural network. Therefore, a result of the down-sampling 105 may have a resolution which is lower than that of each of the input feature map Zk and the first grid data D101.
The result of the down-sampling 105 may be provided to the series of residual blocks 107 and 109. As illustrated in
The 3D feature map D111 may include information about a state of a 3D structure and may be converted into 1D data through the flattening 111 so as to be provided to the network 112. The network 112 may include first to nth layers L1 to Ln (where n is an integer more than 1), and the first to nth layers L1 to Ln may process flattened data or an output of a previous layer. In some embodiments, each of the first to nth layers L1 to Ln may be a fully connected layer or a dense layer. The network 112 may be trained along with the at least one down-sampling based model 85 of
In some embodiments, 3D modeling may replace a simulator for simulating a semiconductor device. For example, the 3D modeling 10 of
Referring to
Referring to
Referring to
Referring to
In operation S120, grid data D12 may be generated. In order to perform 3D modeling based on deep learning, the grid data D12 having a uniform grid may be generated from the geometric data D11. In some embodiments, the grid data D12 may have a grid corresponding to a minimum interval of a grid of the geometric data D11 and may be generated by interpolating the grid of the geometric data D11.
In operation S130, the grid data D12 may be down-sampled. A 3D feature map D16, as described below, generated from the input parameters D15 may have a low resolution. Therefore, the grid data D12 having a high resolution may be progressively down-sampled, whereby a plurality of pieces of down-sampled grid data may be generated.
In operation S140, the 3D feature map D16 may be generated. In order to be provided to a machine learning model along with 3D grid data, 1D input parameters D15 may be pre-processed, and thus, the 3D feature map D16 may be generated.
In operation S150, attribute profile data D17 may be generated. For example, the grid data D12 generated in operation S120, the down-sampled grid data generated in operation S130, and the 3D feature map D16 generated in operation S140 may be provided to at least one up-sampling based model 18. The at least one up-sampling based model 18 may be in a state where the at least one up-sampling based model 18 has been trained to output attribute profile data corresponding to a 3D feature map and grid data and may output attribute profile data D17 representing an attribute of a 3D structure.
In operation S160, the attribute profile data D17 may be interpolated. The attribute profile data D17 generated in operation S150 may have the same grid as that of the grid data D12 generated in operation S120. For an attribute profile compatible with a simulator, the attribute profile data D17 may be interpolated to have the same grid as that of the geometric data D1l based on the geometric data D11.
Referring to
In operation S220, grid data D82 may be generated. In order to perform 3D modeling based on deep learning, the grid data D82 having a uniform grid may be generated from the attribute profile data D81. In some embodiments, the grid data D82 may have a grid corresponding to a minimum interval of a grid of the attribute profile data D81 and may be generated by interpolating the grid of the attribute profile data D81.
In operation S230, the grid data D82 may be down-sampled. In order to generate a below-described 3D feature map D86 having a low resolution, the grid data D82 having a high resolution may be down-sampled, and thus, a plurality of pieces of down-sampled grid data may be generated.
In operation S240, the 3D feature map D86 may be generated. For example, the input parameters D85, the grid data D82 generated in operation S220, and the down-sampled grid data generated in operation S230 may be provided to at least one down-sampling based model 85. The at least one down-sampling based model 85 may be in a state where the at least one down-sampling based model 85 has been trained to output the 3D feature map D86 corresponding to the input parameters D85 and grid data and may output the 3D feature map D86 representing an attribute of a 3D structure.
In operation S250, state data D87 may be generated. For example, the 3D feature map D86 generated in operation S240 may be post-processed, and thus, state data D87 may be generated. In some embodiments, the 3D feature map D86 may be flattened as 1D data and flattened data may pass through a series of fully connected layers, and thus, the state data D87 may be generated.
In operation S260, state profile data D88 may be generated. The state profile data D88 may three-dimensionally represent a state of a 3D structure. Therefore, the 3D feature map D86 generated in operation S240 may be provided to the at least one down-sampling based model 85. The at least one down-sampling based model 85 may generate the state profile data D88 having a high resolution based on the 3D feature map D86 and grid data. In some embodiments, the state profile data D88 may be interpolated to have a non-uniform grid based on the attribute profile data D210 having a non-uniform grid, and thus, state profile data compatible with a simulator may be generated.
Referring to
In [Equation 1], x may denote an output of 3D modeling, u(x) may denote an average of outputs of the 3D modeling, σ2(x) may denote a variance of the outputs of the 3D modeling, and y may denote training data (for example, an output of a simulator). Therefore, an output of 3D modeling including machine learning models which are trained by using a loss function defined as in [Equation 1] may conform to the Gaussian distribution.
In operation S340, a loss function in association with data corresponding to a region except an ROI may be set to zero. For example, as described above with reference to
The computer system 160 may denote an arbitrary system including a general-purpose or special-purpose computer system. For example, the computer system 160 may include a personal computer (PC), a server computer, a laptop computer, and an appliance product. As illustrated in
The at least one processor 161 may execute a program module including an instruction executable by a computer system. The program module may include routines, programs, objects, components, a logic, and a data structure, which perform a certain operation or implement a certain abstract data format. The memory 162 may include a computer system-readable medium of a volatile memory type such as random access memory (RAM). The at least one processor 161 may access the memory 162 and may execute instructions loaded into the memory 162. The storage system 163 may non-volatilely store information, and in some embodiments, may include at least one program product including a program module which is configured to perform training of machine learning models for 3D modeling described above with reference to the drawings. In a non-limiting embodiment, a program may include an operating system (OS), at least one application, other program modules, and other program data.
The network adapter 164 may provide an access to a local area network (LAN), a wide area network (WAN), and/or a common network (for example, Internet). The I/O interface 165 may provide a communication channel corresponding to a peripheral device such as a keyboard, a pointing device, or an audio system. The display 166 may output various pieces of information so that a user check the information.
In some embodiments, training of machine learning models for 3D modeling described above with reference to the drawings may be implemented with a computer program product. The computer program product may include a non-transitory computer-readable medium (or a storage medium) including computer-readable program instructions for allowing the at least one processor 161 to perform image processing and/or training of models. In a non-limiting embodiment, a computer-readable instruction may include an assembler instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine dependent instruction, a micro-code, a firmware instruction, state setting data, or a source code or an object code written in at least one programming language. A computer-readable medium is defined to be any medium that constitutes patentable subject matter under 35 U.S.C. § 101 and excludes any medium that does not constitute patentable subject matter under 35 U.S.C. § 101. Memories described herein are more generally tangible storage mediums for storing data and executable software instructions and are non-transitory during the time software instructions are stored therein. As used herein, the term “non-transitory” is to be interpreted not as an eternal characteristic of a state, but as a characteristic of a state that will last for a period. The term “non-transitory” specifically disavows fleeting characteristics such as characteristics of a carrier wave or signal or other forms that exist only transitorily in any place at any time.
The computer-readable medium may include an arbitrary type of medium for non-temporarily keeping and storing instructions executed by the at least one processor 161 or an arbitrary instruction-executable device. The computer-readable medium may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or an arbitrary combination thereof, but is not limited thereto. For example, the computer-readable medium may include a portable computer disk, a hard disk, RAM, read-only memory (ROM), electrically erasable read only memory (EEPROM), flash memory, static RAM (SRAM), a compact disk (CD), a digital video disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card, or an arbitrary combination thereof.
Referring to
The at least one processor 171 may execute instructions. For example, the at least one processor 171 may execute instructions stored in the memory 173 to execute an OS and may execute applications executed in the OS. In some embodiments, the at least one processor 171 may execute instructions, and thus, the AI accelerator 175 and/or the hardware accelerator 177 may instruct an operation and may obtain a performance result of an operation from the AI accelerator 175 and/or the hardware accelerator 177. In some embodiments, the at least one processor 171 may include an application specific instruction set processor (ASIP) customized for a certain purpose and may support a dedicated instruction set.
The memory 173 may have an arbitrary structure which stores data. For example, the memory 173 may include a volatile memory device such as dynamic RAM (DRAM) or SRAM, and moreover, may include a non-volatile memory device such as flash memory or resistive RAM (RRAM). The at least one processor 171, the AI accelerator 175, and the hardware accelerator 177 may store data (for example, IN, IMG_I, IMG_O, and OUT of
The AI accelerator 175 may denote hardware designed for AI applications. In some embodiments, the AI accelerator 175 may include a neural processing unit (NPU) for implementing a neuromorphic structure, and moreover, may process input data provided from the at least one processor 171 and/or the hardware accelerator 177 to generate output data and may provide the output data to the at least one processor 171 and/or the hardware accelerator 177. In some embodiments, the AI accelerator 175 may be programmable and may be programmed by the at least one processor 171 and/or the hardware accelerator 177.
The hardware accelerator 177 may be referred to as hardware designed for performing a certain operation at a high speed. For example, the hardware accelerator 177 may be designed to perform data conversion such as demodulation, modulation, encoding, or decoding at a high speed. The hardware accelerator 177 may be programmable and may be programmed by the at least one processor 171 and/or the AI accelerator 175.
In some embodiments, the AI accelerator 175 may execute machine learning models described above with reference to the drawings. For example, the AI accelerator 175 may execute each of the layers described above. The AI accelerator 175 may process an input parameter, a feature map, and/or the like to generate an output including useful information. Also, in some embodiments, at least some of models executed by the AI accelerator 175 may be executed by the at least one processor 171 and/or the hardware accelerator 177.
While the inventive concept(s) described herein have been particularly shown and described with reference to embodiments thereof, it will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the following claims.
Number | Date | Country | Kind |
---|---|---|---|
10-2021-0149022 | Nov 2021 | KR | national |