Simple-Sampling and Hard-Mixup with Prototypes to Rebalance Contrastive Learning for Text Classification
arxiv(2024)
摘要
Text classification is a crucial and fundamental task in natural language
processing. Compared with the previous learning paradigm of pre-training and
fine-tuning by cross entropy loss, the recently proposed supervised contrastive
learning approach has received tremendous attention due to its powerful feature
learning capability and robustness. Although several studies have incorporated
this technique for text classification, some limitations remain. First, many
text datasets are imbalanced, and the learning mechanism of supervised
contrastive learning is sensitive to data imbalance, which may harm the model
performance. Moreover, these models leverage separate classification branch
with cross entropy and supervised contrastive learning branch without explicit
mutual guidance. To this end, we propose a novel model named SharpReCL for
imbalanced text classification tasks. First, we obtain the prototype vector of
each class in the balanced classification branch to act as a representation of
each class. Then, by further explicitly leveraging the prototype vectors, we
construct a proper and sufficient target sample set with the same size for each
class to perform the supervised contrastive learning procedure. The empirical
results show the effectiveness of our model, which even outperforms popular
large language models across several datasets.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要