Regularization of Linear Machine Learning Problems.

In this paper, we consider the simplest version of a linear neural network (LNN). Assuming that for training (constructing an optimal weight matrix Q) we have a set of training pairs, i.e. we know the input data G={g^(1),g^(2),⋯,g^(K)}, as well as the correct answers to these input data H={h^(1),h^(2),⋯,h^(K)}. We will study the possibilities of constructing a weight matrix Q of a neural network that will give correct answers to arbitrary input data based on the connection of the specified problem with a system of linear algebraic equations (SLAE). Consider a class of neural networks in which each neuron has only one output signal and performs linear operations. We will show how such LNEs are reduced to SLAEs. Since the questions G and the correct answers H are known to us, the desired weight matrix Q must satisfy the equations Qg^(k)=h^(k), k=1,2,⋯,K. It is required to restore Q. In the general case, the matrix Q is rectangular Q=Q_MN={q_mn}, m is the row number, and g^(k)∈ℝ^N, h^(k)∈ℝ^M. Let G_NK be a matrix composed of columns g^(1), g^(2),⋯,g^(k), and H_MK be a matrix composed of columns h^(1),h^(2),⋯,h^(k). Then, with respect to Q_MN, we obtain a matrix SLAE. Q_MNG_NK=H_MK. This paper will present methods for regularizing the constructed system.