Learning Label Hierarchy with Supervised Contrastive Learning
CoRR(2024)
摘要
Supervised contrastive learning (SCL) frameworks treat each class as
independent and thus consider all classes to be equally important. This
neglects the common scenario in which label hierarchy exists, where
fine-grained classes under the same category show more similarity than very
different ones. This paper introduces a family of Label-Aware SCL methods
(LASCL) that incorporates hierarchical information to SCL by leveraging
similarities between classes, resulting in creating a more well-structured and
discriminative feature space. This is achieved by first adjusting the distance
between instances based on measures of the proximity of their classes with the
scaled instance-instance-wise contrastive. An additional instance-center-wise
contrastive is introduced to move within-class examples closer to their
centers, which are represented by a set of learnable label parameters. The
learned label parameters can be directly used as a nearest neighbor classifier
without further finetuning. In this way, a better feature representation is
generated with improvements of intra-cluster compactness and inter-cluster
separation. Experiments on three datasets show that the proposed LASCL works
well on text classification of distinguishing a single label among
multi-labels, outperforming the baseline supervised approaches. Our code is
publicly available.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要