Patent Application
20180113967
The field of materials science and engineering can involve conducting experiments and simulations to understand the science of materials in order to discover and engineer new materials. Over the last few years, data generated by experiments and simulations has grown exponentially. In the field of materials science, this has led to the emergence of a new field called materials informatics, which deciphers the processing-structure-property-performance (PSPP) relationships in materials science.
In June 2011, the US government launched the Materials Genome Initiative (MGI) to realize the vision of development of advanced materials necessary for economic security and human well-being. The MGI Strategic Plan released in 2014 identifies data analytics as an objective as part of integrating experiments, computation and theory. MGI-supported efforts and other similar efforts around the world are promoting the availability and accessibility of digital data in materials science.
The systems and methods include a predictor module and related systems for predicting a material property, e.g., a field strength, of alloys. For purposes of explanation, the systems and methods are described for predicting a fatigue strength of steel alloys, but other metals may be used. Fatigue strength is a mechanical property of steel and the predictor module can be used to predict other properties too, based on an implementation. In some examples, the fatigue strength of steel alloys is represented by their composition and processing information. In some examples, the PSPP relationships, predicting property of a material given its composition and processing parameters, are built in accordance with observations from the Japan National Institute of Materials Science (NIMS) steel fatigue dataset. Other databases can be used. Modeling techniques, including but not limited to ensemble modeling, can identify models for different attribute sets. In some examples, data-driven feature selection techniques can be used to find a small non-redundant subset of composition and processing attributes for the predictive module. In some examples, the predictive module can be deployed in as an online web-tool, e.g., available at http://info.eecs.northwestern.edu/SteelFatigueStrengthPredictor, or mobile application (example screenshots in
The user devices 102 can be used to input composition parameters and/or processing parameters of the alloys to the predictor module 104, and send the parameters to the predictor module 104 which uses to make alloy fatigue strength predictions. The user devices 102 can also receive a predicted fatigue strength of the alloy from the predictor module 104 and display the predicted fatigue strength to users. The predictor module 104 can also connect with one or more databases 108a-b to obtain data used for making the strength predictions, as described in more detail below. In some examples, the databases include the Japan NIMS steel fatigue dataset. Other databases can be used. The predictor module 104 can receive and store data used for the predictions locally in database 110 for processing and/or process the data stored in the databases 108a-d without the need to store is locally. In some examples, the predictor module 104 mines the data stored in the databases 108a-d, e.g., sorts through the datasets to identify patterns and establish relationships to predict alloy strength through data analysis, e.g., based on the composition parameters and/or processing parameters received from the user devices 102.
For purposes of explanation, the predictor module 104 predicts a fatigue strength of an inputted steel alloy, which can help save a cost and time of fatigue testing, and potentially negative consequences of fatigue failures. Fatigue strength can provide important information for design and failure analysis of mechanical components. In some examples, fatigue strength is estimated to account for over 90% of all mechanical failures of structural components. As described in more detail below, in some examples, the predictor module 104 uses only composition attributes and processing attributes to predict strength. In some examples, the predictor module 104 provides a reduced set of non-redundant attributes to require fewer inputs while still providing a determined predictive accuracy. In some examples, the predictor module 104 can be used by materials science and engineering communities to make fast and accurate predictions of the fatigue property of steel or other alloys, which can aid in discovering better steels and other alloys.
In some examples, the predictor module 104 can perform feature selection to identify a reduced, non-redundant subset of attributes to be used as inputs for prediction, so that the user devices 102 do not have to enter a lot of values (208). The fatigue strength prediction parameters 210 can be stored in database 110. The predictor module 104 can use the prediction parameters 210 for predictive modeling (212). The predictor module 104 can use learning techniques on dataset 214 to learn predictive models for fatigue strength. The predictor module 104 can evaluate the predictive models 212 using validation techniques (216), and deploy the accurate models to the user devices 102, e.g., in an online web-tool and/or mobile application that can predict the fatigue strength of arbitrary inputted compositions and processing parameters (218).
In some examples, the predictor module 104 accesses the fatigue dataset for steel from the Japan NIMS MatNavi to obtain the raw fatigue strength data 202. The Japan NIMS MatNavi is a large database with details on composition, mill product (upstream) features and subsequent processing (heat treatment) parameters. The Japan NIMS MatNavi includes information on carbon and low-alloy steels, carburizing steels and spring steels, for example. Apart from composition and processing details, the Japan NIMS MatNavi also includes data on mechanical properties of steels, e.g., rotating bending fatigue strength at 10′ cycles at room temperature conditions. Fatigue strength includes the highest stress that a material can withstand for a given number of cycles without breaking, and is thus can be an important property of steel for industrial use. The features in the NIMS dataset can be categorized into the following: chemical composition—% C, % Si, % Mn, % P, % S, % Ni, % Cr, % Cu, % Mo (all in wt. %); upstream processing details, e.g., ingot size, reduction ratio, non-metallic inclusions; heat treatment conditions, e.g., temperature, time and other process conditions for normalizing, through-hardening, carburizing, quenching and tempering processes; and mechanical property, e.g., fatigue strength (MPa).
The raw dataset 202 from NIMS includes multiple grades of steel and in some records, some of the heat treatment processing steps do not exist. This is because different specimens can be subjected to different processing routes where some processing steps may not have occurred. To make a coherent database, the predictor module 104 can include the processes in the data, e.g., normalization, through hardening, carburization, quenching, and tempering. For the cases where a given process did not take place, the predictor module 104 can set a corresponding time variable to zero and set the corresponding temperature to the austenization temperature or the average of rest of the data where the process exists. The preprocessed data 204 can also be made publicly available as supplementary to the raw data 202.
In some examples, the preprocessed data 204 can include hundreds of instances/rows and dozens of features/columns (composition and processing parameters), and a single target property (fatigue strength). The details of the attributes can include: C—% Carbon, Si—% Silicon, Mn—% Manganese, P—% Phosphorus, S—% Sulphur, Ni—% Nickel, Cr—% Chromium, Cu—% Copper, Mo—% Molybdenum, NT—Normalizing Temperature, THT—Through Hardening Temperature, THt—Through Hardening Time, THQCr—Cooling Rate for Through Hardening, CT—Carburization Temperature, Ct—Carburization Time, DT—Diffusion Temperature, Dt—Diffusion time, QmT—Quenching Media Temperature (for Carburization), TT—Tempering Temperature, Tt—Tempering Time, TCr—Cooling Rate for Tempering, RedRatio—Reduction Ratio (Ingot to Bar), dA—Area Proportion of Inclusions Deformed by Plastic Work, dB—Area Proportion of Inclusions Occurring in Discontinuous Array, dC—Area Proportion of Isolated Inclusions, and Fatigue—Rotating Bending Fatigue Strength (e.g., 107 Cycles).
In some examples, the predictor module 104 can use a correlation feature selection (CFS) method for feature ranking. CFS is used to identify a subset of features highly correlated with the class variable and weakly correlated amongst them. The predictor module 104 can use CFS in conjunction with a best first search to find a subset S with best average merit, which is given by:
where n is the number of features in S,
The predictive modeling 212 can include regression schemes, including both direct application of regression techniques and constructing their ensembles using various ensembling techniques, including, but not limited to, one or more of 1) linear regression, 2) nearest-neighbor (IBk), 3) nearest-neighbor (KStar), 4) artificial neural networks, 5) Gaussian process, 6) support vector machines, 7) decision tables, 8) decision stumps, 9) M5 model trees, 10) random trees, 11) reduced error pruning tree, 12) random forest, 13 additive regression, 14) bagging, 15) random committee, 16) random subspace, 17) rotations forest and/or 18) voting. Linear regression represents a regression that is linear in the unknown parameters used in the fit. Least squares fitting of lines and polynomials are both forms of linear regression. Nearest-neighbor (IBk), also known as instance-based model, uses normalized Euclidean distance to find the training instance closest to the given test instance, and predicts the same target value as this training instance. If multiple instances have the same (smallest) distance to the test instance, the first one found is used. It eliminates the need for building models and supports adding new instances to the training database dynamically. Nearest-neighbor (KStar) is another type of nearest-neighbor model that uses an entropy-based distance function instead of Euclidean distance.
Artificial neural networks (ANNs) are networks of interconnected artificial neurons, and are used for non-linear statistical data modeling to model complex relationships between inputs and outputs. The network includes a hidden layer of multiple artificial neurons connected to the inputs and outputs with different edge weights. The internal edge weights are learnt during the training process using techniques like back propagation. In some examples, the predictor module 104 uses multilayer perceptron (MLP) for regression with one hidden layer. A Gaussian Process generates data located throughout some domain such that any finite subset of the range follows a multivariate Gaussian distribution, and uses that distribution to make predictions. Support vector machines (SVMs) are based on the Structural Risk Minimization (SRM) principle from statistical learning theory. In some examples, the predictor module 104 uses SVMs for regression. SVMs attempt to perform modeling by constructing hyperplanes in a multidimensional space that separates the instances according to the target variable. SMVs support both classification and regression tasks and can handle multiple continuous and nominal variables.
Decision tables construct rules involving different combinations of attributes, which are selected using an attribute selection search method. A decision stump is a weak tree-based machine learning model including a single-level decision tree with a categorical or numeric class label. Decision stumps are usually used in ensemble machine learning techniques. M5 Model Trees are a reconstruction of Quinlan's M5 algorithm for inducing trees of regression models, which combines a conventional decision tree with the option of linear regression functions at the nodes. M5 Model Trees try to partition the training data using a decision tree induction algorithm by trying to minimize the intra-subset variation in the class values down each branch, followed by back pruning and smoothing, which increases prediction performance. M5 Model Trees also uses the techniques used in CART to effectively deal with enumerated attributes and missing values. Random Tree is a decision tree model that considers a randomly chosen subset of attributes at each node. The number of attributes chosen are, in general, significantly less than the total number of attributes. Random trees are usually used as building blocks for random forests, which, in general, has been found to improve prediction performance. Reduced error pruning tree, known as REPTree, is an implementation of a fast decision tree learner, which builds a decision/regression tree using information gain/variance and prunes it using reduced-error pruning to avoid over-fitting. Part of the training data is withheld from decision tree construction as a pruning set and is subsequently used for pruning. At each internal node in the tree, an error rate is identified by propagating the errors upwards from the leaf nodes. This is compared to the error rate if that internal node was replaced by a leaf node with the average value of the target attribute in that node. If it results in a reduction of error, then the subtree below the node can be pruned, and the node with the highest scope of reducing error is pruned. The random forest model includes multiple decision trees. In that sense, it is an ensemble of random trees. The final prediction of an instance in a random forest is given by the average of the predictions from the individual trees. In many cases, the random forest is known to produce robust and accurate predictions, along with the ability to handle a very large number of input variables, while also being relatively robust to over-fitting.
Additive regression is a meta learner that enhances the performance of a regression base classifier. Each iteration fits a model to the residuals left by the classifier on the previous iteration. The predictions of each of the learners are added together to get the overall prediction. Bagging is an ensemble learning algorithm to improve the stability of classification and regression algorithms by reducing variance. Bagging is usually applied to decision tree models to boost their performance. It involves generating a number of new training sets, called bootstrap modules, from the original set by sampling uniformly with replacement. The bootstrap modules are then used to generate models whose predictions are averaged to generate the final prediction. Bagging has been shown to work better with decision trees than with linear models. Random Committee is a technique for building an ensemble of randomizable base models. Each base model is built using a different random seed but uses the exact same data. The final prediction is a simple average of the individual predictions. The random subspace ensembling technique constructs a decision tree based model consisting of multiple trees, which are constructed systematically by pseudo-randomly selecting subsets of features, trying to achieve a balance between overfitting and achieving maximum accuracy. It maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity.
Rotation Forest is a method for generating model ensembles based on feature extraction, which can work both with classification and regression base learners. Training data for the base modeling technique is created by applying Principal Component Analysis (PCA) to K subsets of the feature set, followed by K axis rotations to form the new features for the base learner, to encourage simultaneously individual accuracy and diversity within the ensemble. Voting is a popular ensemble technique for combining multiple classifiers. Ensemble classifiers using voting may outperform the individual classifiers in certain cases. The predictor module 104 can combine multiple classifiers by using the average of predictions generated by each model, although the predictions can be combined in other ways, such as taking the maximum, minimum, median, etc.
In some examples, the predictor module 104 obtains different modeling configurations using the above techniques as follows. For example, the predictor module 104 uses techniques 1-12 on the training data to get predictive models. The five ensembling techniques, techniques 13-17, work in conjunction with a base modeling technique. In some examples, the predictor module 104 excluded the two nearest-neighbor models, Gaussian process, SVM, decision table, and random forest models while ensembling for one or more of the following reasons: large model size, large training/testing time, low accuracy, already an ensemble model. Of five ensemble methods, random committee works with randomizable base models, e.g., which use a random seed to build a model. Three of the remaining direct modeling techniques fulfill that criterion, which are multilayer perceptron, random tree, and reduced error pruning trees. Further, the predictor module 104 can identify a set of best performing models from the above analysis whose performance are not statistically distinguishable at p=0.05, and an ensemble voting model (technique 18) that averages the predictions from the best performing models to generate the final prediction.
The predictor module 104 can use quantitative assessments of the degree to how close the models predict the experimental fatigue strength to evaluate the models' predictive performance (216). Metrics used for this purpose include the coefficient of correlation (R), explained variance (R2), Mean Absolute Error (MAE), Root Mean Squared Error (RAISE), Relative Absolute Error (RAE), and Root Relative Squared Error (RRSE). Formulae of these evaluation criteria are as follows:
where y denotes the actual fatigue strength (MPa), ŷ denotes the predicted fatigue strength (MPa),
In some examples, the predictor module 104 can use a cross validation setting to evaluate the models, which randomly divides the dataset 214 into multiple parts, e.g., 9 parts as training set and 1 part as the test set, and repeats the process a determined number of times with different test sets before aggregating the results together. Therefore, each labeled instance in the steel fatigue strength prediction database is tested once by a model that did not see it while training. The predictor module 104 can repeat the entire process a determined number of times to aid in statistical significance testing. The predictor module 104 determined optimizations and validations can be used to determine inputs for the input screen 600 (e.g.,
Table 1 is an example comparison of different techniques with 10-fold cross-validation setting using all attributes (table sorted by MAE, where example best accuracy numbers are boldfaced that were statistically not distinguishable at p=0.05, and modeling techniques used in final Voting model also are boldfaced).
RotationForest
—
M5
0.9900
0.9801
18.74
26.50
14.76
14.44
RotationForest
—
MLP
0.9894
0.9789
18.97
27.00
15.00
14.76
Bagging
—
MLP
0.9895
0.9791
18.97
27.03
14.99
14.78
AdditiveRegression
—
M5
0.9897
0.9795
19.05
26.66
15.01
14.54
The example training and testing times for each model, and the model size is listed. Since the predictor module 104 can perform a 10-fold cross-validation, training time is on 90% of the data, testing time is on 10% of the data, and model size also corresponds to the model build on 90% of the data (averaged across a determined number of runs, e.g., 100 runs). In some examples, the predictor module 104 can use Waikato Environment for Knowledge Analysis (WEKA) data mining software version 3.7.13 for analytics with default parameters, unless otherwise stated. The example results were obtained by using the entire set of input attributes. Table 1 is sorted by the MAE metric, and the performance numbers that are not statistically distinguishable at p=0.05 are boldfaced. The top four models from Table 1 are subsequently combined using the Voting modeling scheme to obtain the final model (R2=0.9819, MAE=17.67 MPa, RMSE=25.08 MPa), whose performance is found to be better than all the four constituent models, as well as better than the modeling techniques used previously in A. Agrawal, P. D. Deshpande, A. Cecen, G. P. Basavarsu, A. N. Choudhary, S. R. Kalidindi, “Exploration of data science techniques to predict fatigue strength of steel from composition and processing parameters,” Integrating Materials and Manufacturing Innovation 3 (8), 2014, pp. 1-19, at p=0.05.
Since the NIMS dataset can provide composition and processing attributes, additional examples can be performed with composition attributes (9) and processing attributes (16). The same setting of 10-fold cross-validation can be used, with 10 runs for statistical significance testing. Additive regression with M5 model trees as underlying regressor can be found to be the most accurate model for the composition-only dataset (R2=0.9308, MAE=38.86 MPa, RMSE=48.14 MPa), and also better than other models, so a Voting scheme may not be necessary to combine multiple models. For the processing-only dataset, two models can result in statistically indistinguishable performance. One is RandomForest and the other is RandomCommittee with REPTree as the base regressor. Combining these two with Voting scheme can provide the following accuracy numbers: R2=0.9738, MAE=21.63 MPa, RMSE=30.19 MPa. In these examples, neither composition attributes alone nor processing attributes alone perform as well as using both together, suggesting that they capture complimentary information about materials, and can contribute to model accuracy.
In another example, correlation feature selection (CFS) can be used to identify subsets of both kinds of composition and processing attributes. The predictor module 104 application of CFS technique to composition attributes can identify a subset of six composition attributes: C, Si, P, Cr, Cu and Mo. The same analysis on processing attributes can identify a subset of three processing attributes: THT (through hardening temperature), THQCr (cooling rate for through hardening), and Tt (tempering time). Combining these six composition and three processing attributes make a new dataset of nine attributes. The predictor module 104 can perform the regression modeling with various modeling schemes using the same settings (10 runs of 10-fold cross-validation) to obtain best predictive models for the example dataset. Table 2 presents an example comparison of different techniques with 10-fold cross-validation setting using reduced subset of 9 non-redundant attributes (table sorted by MAE, best accuracy numbers boldfaced that were statistically not distinguishable at p=0.05, modeling techniques used in the final Voting model also boldfaced). The top three models can be found to have statistically indistinguishable accuracy on all performance metrics, and are thus combined using the Voting scheme, resulting in the following accuracy numbers: R2=0.9440, MAE=36.41 MPa, RMSE=44.14 MPa
RandomCommittee
—
REPTree
0.9680
0.9370
37.36
45.74
29.57
25.04
RotationForest
—
MLP
0.9679
0.9368
37.86
46.18
29.98
25.29
RotationForest
—
REPTree
0.9673
0.9357
37.86
46.41
29.96
25.43
0.9666
0.9343
0.9666
0.9343
0.9669
0.9349
0.9662
0.9335
0.9667
0.9345
Table 3 lists example accuracy numbers of the final Voting models on different subsets of the NIMS database.
To make the predictive models readily accessible for use by the materials science and engineering community, the predictor module 104 can determine steel fatigue strength from inputted values 602, 604 of a reduced set of attributes, e.g., as described above. In some examples, the predictor module 104 generates predictions of fatigue strength for a steel. The reduced set of attributes 600 can be determined for the predictor module 104 so that the user device 102 does not have to enter too many attributes and still obtain a satisfactory accuracy, but the option of using a fuller predictor (700 in
In some example embodiments, the computing device 900 may include processing circuitry 910 that is configurable to perform actions in accordance with one or more example embodiments disclosed herein. In this regard, the processing circuitry 910 may be configured to predict strength of alloys based on received composition parameters and/or processing parameters. The processing circuitry 910 may be configured to perform data mining, data processing, application execution and/or other processing according to one or more examples. In some examples, the computing device 900 or a portion(s) or component(s) thereof, such as the processing circuitry 910, may include one or more chipsets and/or other components that may be provided by integrated circuits.
In some example embodiments, the processing circuitry 910 may include a processor 912 and, in some embodiments, such as that illustrated in
In some example embodiments, the memory 914 may include one or more memory devices. Memory 914 may include fixed and/or removable memory devices. In some embodiments, the memory 914 may provide a non-transitory computer-readable storage medium that may store computer program instructions that may be executed by the processor 912. In this regard, the memory 914 may be configured to store information, data, applications, instructions and/or the like for enabling the computing device 900 to carry out various functions in accordance with one or more example embodiments. In some embodiments, the memory 914 may be in communication with one or more of the processor 912, the user interface 916 for passing information among components of the computing device 900.
While various embodiments have been described, it can be apparent that many more embodiments and implementations are possible. Accordingly, the embodiments are not to be restricted.
This patent application claims the benefit of U.S. Provisional Patent Application No. 62/413,261, filed on Oct. 26, 2016, the entire contents of which is incorporated by reference in its entirety.
This invention was made with government support under FA9550-12-1-0458 awarded by the Air Force Office of Scientific Research and 70NANB14H012 awarded by the National Institute of Standards and Technology (NIST). The government has certain rights in the invention.
| Number | Date | Country | |
|---|---|---|---|
| 62413261 | Oct 2016 | US |
