Towards Perpetually Trainable Neural Networks

Underpinning the past decades of work on the design, initialization, and optimization of neural networks is a seemingly inoccuous assumption: that the networkis trained on a stationary data distribution. In settings where this assumption is violated, e.g. deep reinforcement learning, learning algorithms become unstableand brittle with respect to hyperparameters and even random seeds. One factor driving this instability is the loss of plasticity, meaning that updating the network’s predictions in response to new information becomes more difficult as training progresses. In this paper, we conduct a thorough analysis of the mehcnaisms of plasticity loss in neural networks trained on nonstationary learning problems, identify solutions to each of these pathologies, and integrate these solutions into a straightforward training protocol designed to maintain plasticity. We validate this approach in a variety of synthetic continual learning tasks, and further demonstrate its effectiveness on naturally arising nonstationarities