Task-Adaptive Sparse Update for Efficient Continual Learning

Continual learning (CL) aims to learn tasks sequentially, reflecting real-world applications such as intelligent surveillance systems. Such practical situations are often resource-constrained, making sparse continual learning (sparse CL) particularly attractive as it ensures computational and memory efficiency by updating only a small subset of parameters without model expansion. However, existing sparse CL methods still struggle with the core challenge of CL: the stability–plasticity dilemma—retaining prior knowledge (stability) while acquiring new knowledge (plasticity). This paper proposes adaptive continual sparse learning (ACSL), a novel sparse CL method that adaptively controls both how many and which parameters are updated for each task. ACSL estimates the task familiarity of the model and dynamically allocates the number of parameters to update, thereby tailoring the stability–plasticity balance to each task. Furthermore, ACSL periodically reselects parameters to update in a stochastic manner, controlling plasticity at the parameter level. Experiments on multiple image classification benchmarks show that ACSL achieves higher accuracy than existing sparse CL methods across most datasets.