Contrastive learning from label distribution: A case study on text classification

•The proposed method learns under the supervision of the predicted label distribution.•The two contrastive losses work in the label space instead of the embedding space.•It leads to significant improvements over large label spaces or limited label data.•It captures label correlations better than working in the embedding space.