Using deep neural network with small dataset to predict material defects

Materials & Design, pp. 300-310, 2019.

Cited by: 48|Bibtex|Views61|DOI:https://doi.org/10.1016/j.matdes.2018.11.060
Other Links: academic.microsoft.com
Weibo:
Study in this paper demonstrates that small/narrow deep neural network with small dataset and special training methods has huge potential for extensive applications in material study, especially for those multivariable nonlinear problems

Abstract:

Abstract Deep neural network (DNN) exhibits state-of-the-art performance in many fields including microstructure recognition where big dataset is used in training. However, DNN trained by conventional methods with small datasets commonly shows worse performance than traditional machine learning methods, e.g. shallow neural network and s...More

Code:

Data:

0
Introduction
  • Research papers [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18] and Reviews [19,20] about SNN applications in material science and engineering have been widely reported.
Highlights
  • The success of the previous generation neural network is limited to shallow neural network (SNN) of 1 or 2 hidden layers, because training deep neural network (DNN) is not easy and the resultant accuracy is usually worse than that of SNN [17]
  • DNN can be perceived as an upgrade of SNN
  • Matsuda [75] reported that Mn addition to SUS310S with 0.002 wt% P and 0.004 wt% S exerts a harmful effect on cracking resistance under 4% applied-strain conditions using transverse varestraint test
  • The derived mathematical expression can be used in the material defects prediction, new alloys development, and comparison with experimental results
  • Study in this paper demonstrates that small/narrow DNN with small dataset and special training methods has huge potential for extensive applications in material study, especially for those multivariable nonlinear problems
Methods
  • Training algorithm and Bayesian regularization.
  • The performance function mse and regularized performance function msereg of a neural network are defined as follows: mse 1⁄4 1 N XN ðeiÞ2 i1⁄41.
  • XN i1⁄41 ðt i −ai Þ2.
  • Ð1Þ msereg 1⁄4 γmse þ ð1−γÞmsw ð2Þ msw 1 n Xn W 2 j ð3Þ j1⁄41
Results
  • Matsuda [75] reported that Mn addition to SUS310S with 0.002 wt% P and 0.004 wt% S exerts a harmful effect on cracking resistance under 4% applied-strain conditions using transverse varestraint test.
Conclusion
  • Connected DNN which consists of 3 or more hidden layers shows its advantage over shallow neural network and support vector machine in that it can achieve higher prediction accuracy and better generalization performance.

    Through DNN regression, vast scattered experimental data in the literature can produce simple quantitative expression of specific material property as function of chemistry composition and processing parameters, etc.
  • Deep and narrow neural network are suitable for material problems, such as solidification cracking susceptibility prediction in this paper, and pretraining using stacked auto-encoder is effective and necessary in the DNN regression of numerical small datasets.
  • Study in this paper demonstrates that small/narrow DNN with small dataset and special training methods has huge potential for extensive applications in material study, especially for those multivariable nonlinear problems
Summary
  • Introduction:

    Research papers [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18] and Reviews [19,20] about SNN applications in material science and engineering have been widely reported.
  • Methods:

    Training algorithm and Bayesian regularization.
  • The performance function mse and regularized performance function msereg of a neural network are defined as follows: mse 1⁄4 1 N XN ðeiÞ2 i1⁄41.
  • XN i1⁄41 ðt i −ai Þ2.
  • Ð1Þ msereg 1⁄4 γmse þ ð1−γÞmsw ð2Þ msw 1 n Xn W 2 j ð3Þ j1⁄41
  • Results:

    Matsuda [75] reported that Mn addition to SUS310S with 0.002 wt% P and 0.004 wt% S exerts a harmful effect on cracking resistance under 4% applied-strain conditions using transverse varestraint test.
  • Conclusion:

    Connected DNN which consists of 3 or more hidden layers shows its advantage over shallow neural network and support vector machine in that it can achieve higher prediction accuracy and better generalization performance.

    Through DNN regression, vast scattered experimental data in the literature can produce simple quantitative expression of specific material property as function of chemistry composition and processing parameters, etc.
  • Deep and narrow neural network are suitable for material problems, such as solidification cracking susceptibility prediction in this paper, and pretraining using stacked auto-encoder is effective and necessary in the DNN regression of numerical small datasets.
  • Study in this paper demonstrates that small/narrow DNN with small dataset and special training methods has huge potential for extensive applications in material study, especially for those multivariable nonlinear problems
Tables
  • Table1: The accuracies of SVM, SNN and DNN based on dataset without PCA pre-process and datasets after PCA pre-process
  • Table2: The specifications (wt%) of 304 and 310S stainless steel and the testing parameters used for prediction
  • Table3: Compositions (wt%) of stainless steels used in prediction (varied P and S)
Download tables as Excel
Funding
  • Shuo Feng wishes to acknowledge EPSRC CDT (Grant No: EP/ L016206/1) in Innovative Metal Processing for providing a PhD studentship for this study
Study subjects and analysis
raw input datasets: 21
Principal component analysis (PCA) is known to reduce data dimensions and noise in the input data [71]. For this study, 21 raw input datasets were pre-proceeded by PCA method that resulted in two datasets, one with 15 principle components (percentage of the total variance N99%) and the other with 10 principle components (percentage of the total variance N95%). These two pre-proceeded datasets as well as raw dataset were used as input in the training and testing of SVM, SNN and DNN

Reference
  • L. Guo, H. Roelofs, M.I. Lembke, H.K.D.H. Bhadeshia, Modelling of transition from upper to lower bainite in multi-component system, Mater. Sci. Technol. 33 (4) (2017) 430–437.
    Google ScholarLocate open access versionFindings
  • M. Vasudevan, A.K. Bhaduri, B. Raj, K.P. Rao, Artificial neural network modelling of solidification mode in austenitic stainless steel welds, Mater. Sci. Technol. 23 (4) (2007) 451–459.
    Google ScholarLocate open access versionFindings
  • D.R. Cassar, A.C.P.L.F. de Carvalho, E.D. Zanotto, Predicting glass transition temperatures using neural networks, Acta Mater. 159 (2018) 249–256.
    Google ScholarLocate open access versionFindings
  • R.C. Dimitriu, H.K.D.H. Bhadeshia, C. Fillon, C. Poloni, Strength of ferritic steels: neural networks and genetic programming, Mater. Manuf. Process. 24 (1) (2008) 10–15.
    Google ScholarLocate open access versionFindings
  • S. Malinov, W. Sha, Application of artificial neural networks for modelling correlations in titanium alloys, Mater. Sci. Eng. A 365 (1) (2004) 202–211.
    Google ScholarLocate open access versionFindings
  • W. Solano-Alvarez, M.J. Peet, E.J. Pickering, J. Jaiswal, A. Bevan, H.K.D.H. Bhadeshia, Synchrotron and neural network analysis of the influence of composition and heat treatment on the rolling contact fatigue of hypereutectoid pearlitic steels, Mater. Sci. Eng. A 707 (2017) 259–269.
    Google ScholarLocate open access versionFindings
  • H.K.D.H. Bhadeshia, D.J.C. MacKay, L.E. Svensson, Impact toughness of C–Mn steel arc welds – Bayesian neural network analysis, Mater. Sci. Technol. 11 (10) (1995) 1046–1051.
    Google ScholarLocate open access versionFindings
  • T. Sourmail, H.K.D.H. Bhadeshia, D.J.C. MacKay, Neural network model of creep strength of austenitic stainless steels, Mater. Sci. Technol. 18 (6) (2002) 655–663.
    Google ScholarLocate open access versionFindings
  • G. Allegri, Modelling fatigue delamination growth in fibre-reinforced composites: power-law equations or artificial neural networks? Mater. Des. 155 (2018) 59–70.
    Google ScholarLocate open access versionFindings
  • Z. Anjum, F. Qayyum, S. Khushnood, S. Ahmed, M. Shah, Prediction of nonpropagating fretting fatigue cracks in Ti6Al4V sheet tested under pin-in-dovetail configuration: experimentation and numerical simulation, Mater. Des. 87 (2015) 750–758.
    Google ScholarLocate open access versionFindings
  • B.D. Conduit, N.G. Jones, H.J. Stone, G.J. Conduit, Design of a nickel-base superalloy using a neural network, Mater. Des. 131 (2017) 358–365.
    Google ScholarLocate open access versionFindings
  • M.W. Dewan, D.J. Huggett, T. Warren Liao, M.A. Wahab, A.M. Okeil, Prediction of tensile strength of friction stir weld joints with adaptive neuro-fuzzy inference system (ANFIS) and neural network, Mater. Des. 92 (2016) 288–299.
    Google ScholarLocate open access versionFindings
  • G. Liu, L. Jia, B. Kong, K. Guan, H. Zhang, Artificial neural network application to study quantitative relationship between silicide and fracture toughness of Nb-Si alloys, Mater. Des. 129 (2017) 210–218.
    Google ScholarLocate open access versionFindings
  • O. Palavar, D. Özyürek, A. Kalyon, Artificial neural network prediction of aging effects on the wear behavior of IN706 superalloy, Mater. Des. 82 (2015) 164–172.
    Google ScholarLocate open access versionFindings
  • X. Xia, J.F. Nie, C.H.J. Davies, W.N. Tang, W. Xu, N. Birbilis, An artificial neural network for predicting corrosion rate and hardness of magnesium alloys, Mater. Des. 90 (2016) 1034–1043.
    Google ScholarLocate open access versionFindings
  • L. Aijun, L. Hejun, L. Kezhi, G. Zhengbing, Applications of neural networks and genetic algorithms to CVI processes in carbon/carbon composites, Acta Mater. 52 (2) (2004) 299–305.
    Google ScholarLocate open access versionFindings
  • S. Malinov, W. Sha, J.J. McKeown, Modelling the correlation between processing parameters and properties in titanium alloys using artificial neural network, Comput. Mater. Sci. 21 (3) (2001) 375–394.
    Google ScholarLocate open access versionFindings
  • K. Ichikawa, H.K.D.H. Bhadeshia, D.J.C. MacKay, Model for solidification cracking in low alloy steel weld metals, Sci. Technol. Weld. Join. 1 (1) (1996) 43–50.
    Google ScholarLocate open access versionFindings
  • H.K.D.H. Bhadeshia, Neural networks in materials science, ISIJ Int. 39 (10) (1999) 966–979.
    Google ScholarLocate open access versionFindings
  • H.K.D.H. Bhadeshia, R.C. Dimitriu, S. Forsik, J.H. Pak, J.H. Ryu, Performance of neural networks in materials science, Mater. Sci. Technol. 25 (4) (2009) 504–510.
    Google ScholarLocate open access versionFindings
  • I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT press, Cambridge, 2016.
    Google ScholarFindings
  • J. Schmidhuber, Deep learning in neural networks: an overview, Neural Netw. 61 (2015) 85–117.
    Google ScholarFindings
  • G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science 313 (5786) (2006) 504–507.
    Google ScholarLocate open access versionFindings
  • Y. Lecun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436.
    Google ScholarLocate open access versionFindings
  • G.E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets, Neural Comput. 18 (7) (2006) 1527–1554.
    Google ScholarLocate open access versionFindings
  • Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks, Adv. Neural Inf. Proces. Syst. (2007) 153–160.
    Google ScholarLocate open access versionFindings
  • D. Kotecki, J. Lippold, Welding Metallurgy and Weldability of Stainless Steels, John Wiley & Sons, 2005.
    Google ScholarFindings
  • S. Kou, Welding Metallurgy, John Wiley & Sons, 2003.
    Google ScholarFindings
  • J. Campbell, Chapter 8 - Cracks and tears, Complete Casting Handbook, Butterworth-Heinemann, Oxford, 2011 465–497.
    Google ScholarLocate open access versionFindings
  • J.A. Dantzig, M. Rappaz, Solidification, EPFL press, 2009.
    Google ScholarFindings
  • S. Li, D. Apelian, Hot tearing of aluminum alloys, Int. J. Met. 5 (1) (2011) 23–40.
    Google ScholarLocate open access versionFindings
  • W.J. Sames, F.A. List, S. Pannala, R.R. Dehoff, S.S. Babu, The metallurgy and processing science of metal additive manufacturing, Int. Mater. Rev. 61 (5) (2016) 315–360.
    Google ScholarLocate open access versionFindings
  • D.D. Gu, W. Meiners, K. Wissenbach, R. Poprawe, Laser additive manufacturing of metallic components: materials, processes and mechanisms, Int. Mater. Rev. 57 (3) (2012) 133–164.
    Google ScholarLocate open access versionFindings
  • L. Aucott, D. Huang, H.B. Dong, S.W. Wen, J. Marsden, A. Rack, A.C.F. Cocks, A threestage mechanistic model for solidification cracking during welding of steel, Metall. Mater. Trans. A 49 (5) (2018) 1674–1682.
    Google ScholarLocate open access versionFindings
  • L. Aucott, D. Huang, H. Dong, S. Wen, J. Marsden, A. Rack, A. Cocks, Initiation and growth kinetics of solidification cracking during welding of steel, Sci. Rep. 7 (2017), 40255.. [36] D.G. Eskin, L. Katgerman Suyitno, Mechanical properties in the semi-solid state and hot tearing of aluminium alloys, Mater. Sci. Eng. R 49 (5) (2004) 629–711.
    Google ScholarLocate open access versionFindings
  • [37] T. Soysal, S. Kou, A simple test for assessing solidification cracking susceptibility and checking validity of susceptibility prediction, Acta Mater. 143 (2018) 181–197.
    Google ScholarLocate open access versionFindings
  • [38] S. Kou, A criterion for cracking during solidification, Acta Mater. 88 (2015) 366–374.
    Google ScholarLocate open access versionFindings
  • [39] M. Rappaz, J.M. Drezet, M. Gremaud, A new hot-tearing criterion, Metall. Mater. Trans. A 30 (2) (1999) 449–455.
    Google ScholarLocate open access versionFindings
  • [40] C. Bordreuil, A. Niel, Modelling of hot cracking in welding with a cellular automaton combined with an intergranular fluid flow model, Comput. Mater. Sci. 82 (2014) 442–450.
    Google ScholarLocate open access versionFindings
  • [41] A. Niel, C. Bordreuil, F. Deschaux-Beaume, G. Fras, Modelling hot cracking in 6061 aluminium alloy weld metal with microstructure based criterion, Sci. Technol. Weld. Join. 18 (2) (2013) 154–160.
    Google ScholarLocate open access versionFindings
  • [42] B.G. Thomas, Modeling of hot tearing and other defects in casting processes, ASM Handbook 22 (2009) 362–374.
    Google ScholarLocate open access versionFindings
  • [43] A. Cecen, H. Dai, Y.C. Yabansu, S.R. Kalidindi, L. Song, Material structure-property linkages using three-dimensional convolutional neural networks, Acta Mater. 146 (2018) 76–84.
    Google ScholarFindings
  • [44] B.L. Decost, T. Francis, E.A. Holm, Exploring the microstructure manifold: image texture representations applied to ultrahigh carbon steel microstructures, Acta Mater. 133 (2017) 30–40.
    Google ScholarLocate open access versionFindings
  • [45] R. Kondo, S. Yamakawa, Y. Masuoka, S. Tajima, R. Asahi, Microstructure recognition using convolutional neural networks for prediction of ionic conductivity in ceramics, Acta Mater. 141 (2017) 29–38.
    Google ScholarLocate open access versionFindings
  • [46] E. Kim, K. Huang, S. Jegelka, E. Olivetti, Virtual screening of inorganic materials synthesis parameters with deep learning, npj Comp. Mater. 3 (1) (2017) 53.
    Google ScholarLocate open access versionFindings
  • [47] L. Ward, A. Agrawal, A. Choudhary, C. Wolverton, A general-purpose machine learning framework for predicting properties of inorganic materials, npj Comp. Mater. 2 (2016), 16028..
    Google ScholarLocate open access versionFindings
  • [48] L. Ward, S.C. O'Keeffe, J. Stevick, G.R. Jelbert, M. Aykol, C. Wolverton, A machine learning approach for engineering bulk metallic glass alloys, Acta Mater. 159 (2018) 102–111.
    Google ScholarLocate open access versionFindings
  • [49] Y.T. Sun, H. Bai, M.-Z. Li, W. Wang, Machine learning approach for prediction and understanding of glass-forming ability, J. Phys. Chem. Lett. 8 (14) (2017) 3434–3439.
    Google ScholarLocate open access versionFindings
  • [50] H.B. Demuth, M.H. Beale, O. De Jess, M.T. Hagan, Neural Network design, Martin Hagan, 2014.
    Google ScholarFindings
  • [51] D.J. MacKay, A practical Bayesian framework for backpropagation networks, Neural Comput. 4 (3) (1992) 448–472.
    Google ScholarFindings
  • [52] D.J. MacKay, Bayesian interpolation, Neural Comput. 4 (3) (1992) 415–447.
    Google ScholarFindings
  • [53] Y. Arata, F. Matsuda, S. Saruwatari, Varestraint test for solidification crack susceptibility in weld metal of austenitic stainless steels, Trans. JWRI 3 (1) (1974) 79–88.
    Google ScholarLocate open access versionFindings
  • [54] J.A. Brooks, W.M. Garrison, Weld microstructure development and properties of precipitation-strengthened martensitic stainless steels, Weld. J. 78 (8) (1999) 280S–291S. [55] M.J. Cieslak, W.F. Savage, Weldability and solidification phenomena of cast stainlesssteel, Weld. J. 59 (5) (1980) S136–S146.
    Google ScholarLocate open access versionFindings
  • [56] D.H. Kah, D.W. Dickinson, Weldability of ferritic stainless-steels, Weld. J. 60 (8) (1981) S135–S142.
    Google ScholarLocate open access versionFindings
  • [57] L. Li, R.W. Messler, Effects of phosphorus and sulfur on susceptibility to weld hot cracking in austenitic stainless steels, WRC Bul. 480 (2003).
    Google ScholarLocate open access versionFindings
  • [58] J.C. Lippold, An investigation of weld cracking in alloy 800, Weld. J. 63 (3) (1984) S91–S103.
    Google ScholarLocate open access versionFindings
  • [59] C. Lundin, C. Qiao, Y. Kikuchi, C. Shi, T. Gill, Investigation of Joining Techniques for Advanced Austenitic Alloys, Oak Ridge National Lab, TN (United States); Tennessee Univ., Knoxville, TN (United States), 1991.
    Google ScholarFindings
  • [60] C.D. Lundin, C.H. Lee, C.Y.P. Qiao, Evaluation of backfilled solidification cracks in austenitic stainless welds in relationship to evaluation of hot cracking, Weld. J. 72 (7) (1993) S321–S328.
    Google ScholarLocate open access versionFindings
  • [61] C.D. Lundin, C. Qiao, C. Lee, G. Batten, Weldability and hot ductility behavior of nuclear grade austenitic stainless steels, WRC Bul. 509 (2006).
    Google ScholarLocate open access versionFindings
  • [62] C.D. Lundin, C.Y.P. Qiao, T.P.S. Gill, G.M. Goodwin, Hot ductility and hot cracking behavior of modified 316 stainless-steels designed for high-temperature service, Weld. J. 72 (5) (1993) S189–S200.
    Google ScholarLocate open access versionFindings
  • [63] H. Madarame, T. Sukegawa, H. Inoue, Cracking susceptibility of stainless-steel subjected to plasma disruption, Fusion Eng. Des. 27 (1995) 499–506.
    Google ScholarLocate open access versionFindings
  • [64] T. Ogawa, T. Koseki, Weldability of newly developed austenitic alloys for cryogenic service.
    Google ScholarFindings
  • 2. High-nitrogen stainless-steel weld metal, Weld. J. 67 (1) (1988) S8–S17.
    Google ScholarFindings
  • [65] T. Ogawa, E. Tsunetomi, Hot cracking susceptibility of austenitic stainless-steels, Weld. J. 61 (3) (1982) S82–S93.
    Google ScholarLocate open access versionFindings
  • [66] A.M. Ritter, W.F. Savage, Solidification and solidification cracking in nitrogenstrengthened austenitic stainless-steels, Metall. Trans. A. 17 (4) (1986) 727–737.
    Google ScholarLocate open access versionFindings
  • [67] C.V. Robino, J.R. Michael, M.C. Maguire, The solidification and welding metallurgy of galling-resistant stainless steels, Weld. J. 77 (11) (1998) 446S–457S.
    Google ScholarLocate open access versionFindings
  • [68] S. Shono, O. Matsumoto, S. Kawaguchi, T. Urabe, Weldability of 18-percent chromium ferritic stainless-steel with titanium addition, Trans. Iron Steel Inst. Jpn. 24 (7) (1984) 557–565.
    Google ScholarLocate open access versionFindings
  • [69] I. Varol, W.A. Baeslack, J.C. Lippold, Characterization of weld solidification cracking in a duplex stainless-steel, Metallography 23 (1) (1989) 1–19.
    Google ScholarFindings
  • [70] M. Zaghloul, A. Sadek, A. El-Batahgy, M. Hanafy, Effect of welding parameters on hot cracking susceptibility of alloy 800, Yosetsu Gakkai Ronbunshu 12 (3) (1994) 335–341.
    Google ScholarLocate open access versionFindings
  • [71] R. May, G. Dandy, H. Maier, Review of input variable selection methods for artificial neural networks, Artificial Neural Networks-methodological Advances and Biomedical Applications, InTech, 2011.
    Google ScholarLocate open access versionFindings
  • [72] Statistics and Machine Learning ToolboxTM User's Guide., The MathWorks, Inc., 2018
    Google ScholarFindings
  • [73] A.K. Bhaduri, S.K. Albert, B. Raj, 2 - robust welding technologies for ferrous alloys, in: R. Singh (Ed.), Weld Cracking in Ferrous Alloys, Woodhead Publishing 2009, pp. 34–95.
    Google ScholarLocate open access versionFindings
  • [74] Y. Arata, F. Matsuda, H. Nakagawa, S. Katayama, Solidification crack susceptibility in weld metals of fully austenitic stainless steels (report IV): effect of decreasing P and Son solidification crack susceptibility of SUS 310S austenitic stainless steel weld metals, Trans. JWRI 7 (2) (1978) 169–172.
    Google ScholarLocate open access versionFindings
  • [75] F. Matsuda, H. Nakagawa, S. Katayama, Y. Arata, Solidification crack susceptibility in weld metals of fully austenitic stainless steels (report VII): effect of Mn and N on solidification crack resistance, Trans. JWRI 11 (2) (1982) 79–85.
    Google ScholarLocate open access versionFindings
  • [76] F. Matsuda, S. Katayama, Y. Arata, Solidification crack susceptibility in weld metals of fully austenitic stainless steels (report IX): effect of titanium on solidification crack resistance, Trans. JWRI 12 (2) (1983) 247–252.
    Google ScholarLocate open access versionFindings
  • [77] F. Matsuda, H. Nakagawa, S. Katayama, Y. Arata, Solidification crack susceptibility in weld metals of fully austenitic stainless steels (report VIII): effect of nitrogen on cracking in SUS 304 weld metal, Trans. JWRI 12 (6) (1983) 89–95.
    Google ScholarLocate open access versionFindings
  • [78] Y. Arata, F. Matsuda, S. Katayama, Solidification crack susceptibility in weld metals of fully austenitic stainless steels (report II): effect of ferrite, P, S, C, Si and Mn on ductility properties of solidification brittleness, Trans. JWRI 6 (1) (1977) 105–116.
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments