SGDOL: Self-evolving Generative and Discriminative Online Learning for Data Stream Classification

Data streams are usually non-stationary obtained from the same or different data sources. It needs to be processed sequentially, hence termed stream processing. Stream processing often demands evolving neural network architectures that can alter the number of nodes and layers on-demand to classify data streams in an online manner, known as evolving online learning. Traditional deep neural networks (DNNs) uses batch data processing, often limited by their static network structures and offline learning approaches while addressing data streams. In this work, we propose a novel evolving deep neural network framework, known as Self-evolving Generative and Discriminative Online Learning (SGDOL), which utilises an online learning approach to evolve both generator and discriminator network structure from scratch, and on-demand to classify data streams. The dynamic feature learning mechanism of autoencoder-based generative models have demonstrated its potential in learning latent feature representations from data streams. These latent features are fed to the evolving feed-forward DNN-based discriminator as input. The mechanism of adding or pruning nodes in the evolving architecture of discriminator supports in dealing with catastrophic forgetting problems; a new layer is added to the discriminator when a new concept appears in the data stream. To back these theoretical contributions of SGDOL, experiments were conducted using nine benchmark datasets and compared with ten state-of-the-art online learning algorithms. SGDOL performance measure of testing classification rates was better in seven datasets out of nine than the existing algorithms, which clearly indicates its ability to deal with the data stream.