Efficient Global Optimization Algorithm Using Neural Network-based Prediction and Uncertainty

View Video Presentation: https://doi.org/10.2514/6.2023-2683.vid The efficient global optimization (EGO) algorithm (also called Bayesian optimization) has been successfully applied to simulation-based engineering design optimization problems which are severely limited by time. The EGO algorithm is nearly always used in combination with Gaussian process regression (GPR) (also called Kriging) since it provides a prediction model as well as an uncertainty model. The computational cost of GPR, however, grows quickly with the number of samples, which can limit it to design problems with a few number of variables. EGO is not restricted to GPR and the only requirement is a surrogate with an uncertainty model. Deep neural networks (DNNs) have been shown to be capable of handling a large number of samples. In this paper, a new EGO algorithm using DNN-based prediction and uncertainty (EGONN) is proposed. Given an initial set of samples, EGONN iteratively constructs two DNNs, one for the function prediction and the other for the prediction uncertainty, and utilizes them to find the next sample within the design space with the expected improvement infill criterion and, subsequently, update the DNN function prediction model. In this work, the DNN-based prediction uncertainty uses a separate, static training data set. The proposed EGONN algorithm is demonstrated on two analytical test problems, a bounded one-dimensional minimization problem and a constrained two-dimensional minimization problem, and compared with the EGO algorithm. The results show that EGONN yields comparable results as EGO with same number of initial and infill sample budget.