Frame Regularization of a Convolutional Neural Network in Image-Classification Problems

The problem of regularization of the parameters of a neural network is considered in order to increase the efficiency of using their redundancy and increase resistance to implementations of input data that are not contained in the training set. A representation of the system of weight vectors of the neural-network layer as a frame in the space of weights is proposed and regularization is introduced in the form of a penalty for noncompliance with the sufficient condition of the frame. The proposed method imposes less restrictions on the weights of the model than existing methods for increasing efficiency based on orthogonalization. The method is generalized to convolutional layers in the block-Toeplitz representation and is applicable to convolutional neural networks. A computational experiment on CIFAR-10, CIFAR-100 and SVHN datasets shows the superiority of the proposed regularization method in terms of classification accuracy, generalization ability and resistance to adversarial attacks compared to the basic approaches.