Shallow Physics Informed Neural Networks Using Levenberg-Marquardt Optimization

There is a renewed interest in exploring the application of Artificial Neural Networks (ANNs) to solve Differential equations. One of the popular methods is Physics informed Neural Networks(PINNs), which embeds the knowledge of the equation itself into the loss function of the Neural-Network. The traditional PINNs use Multi-layer ANNs (MLNNs) and employ GradientDescent type optimization algorithms like Adam/L-BFGS-B to optimize the weights of the ANN. In this paper, we explore the well-known Levenberg-Marquardt (LM) Optimization algorithm to optimize the weights of a Single-layer Neural Network (SLNN) based PINNs. We show that for a class of problems known as Singular Perturbation Problems (SPPs), our method can achieve much more accurate solutions, much faster, than the Traditional PINNs. The prevalent research on ANNs mostly focuses on the architecture and the data. Based on our observations, we establish that the choice of weight optimization algorithms are as important as the other two and need due consideration.