Adaptive One-Pass Passive-Aggressive Radial Basis Function for Classification Problems

This paper presents a novel adaptive one-pass Passive-Aggressive Radial Basis Function (APARBF) for classification problems. The APARBF uses elliptic Gaussian neurons followed by a Softmax layer and cross-entropy loss function. This network tries to overcome the elasticity-plasticity dilemma by using the Passive-Aggressive (PA) algorithm and adapting the hidden layer structure. The weight updates have to be plastic to acquire the most information from each sample and at the same time need to be elastic to retrain the information from the past instances. Inspired by PA, a novel update formula for cross-entropy loss minimization has been derived. The adaptive design of the network lets it start with zero hidden neurons and grow or shrink according to the data. For kernel parameters, the adaptive structure determines the correct number of hidden neurons and updates recursively each neuron’s center and covariance matrix. To evaluate our network, we perform two series of experiments. The first experiments compare the proposed APARBF with other recently developed one-pass algorithms (i.e., OVIG, OBHT, SCW, AROW, OGD, and PA). The subsequent experiments include comparing the proposed algorithm with some online adaptive structures such as FGAP-RBF, C-Mantec, McNN, and PBL-McNN, based on their mean classification error and the number of hidden neurons. Wilcoxon sign rank test and Friedman test clearly show the superiority of the proposed network’s results compared with its competitors in one-pass classification problems.