Generative Multi-modal Models are Good Class Incremental Learners
CVPR 2024(2024)
摘要
In class-incremental learning (CIL) scenarios, the phenomenon of catastrophic
forgetting caused by the classifier's bias towards the current task has long
posed a significant challenge. It is mainly caused by the characteristic of
discriminative models. With the growing popularity of the generative
multi-modal models, we would explore replacing discriminative models with
generative ones for CIL. However, transitioning from discriminative to
generative models requires addressing two key challenges. The primary challenge
lies in transferring the generated textual information into the classification
of distinct categories. Additionally, it requires formulating the task of CIL
within a generative framework. To this end, we propose a novel generative
multi-modal model (GMM) framework for class-incremental learning. Our approach
directly generates labels for images using an adapted generative model. After
obtaining the detailed text, we use a text encoder to extract text features and
employ feature matching to determine the most similar label as the
classification prediction. In the conventional CIL settings, we achieve
significantly better results in long-sequence task scenarios. Under the
Few-shot CIL setting, we have improved by at least 14% accuracy over all the
current state-of-the-art methods with significantly less forgetting. Our code
is available at .
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要