Overcoming Dimensional Collapse in Self-supervised Contrastive Learning for Medical Image Segmentation
CoRR(2024)
摘要
Self-supervised learning (SSL) approaches have achieved great success when
the amount of labeled data is limited. Within SSL, models learn robust feature
representations by solving pretext tasks. One such pretext task is contrastive
learning, which involves forming pairs of similar and dissimilar input samples,
guiding the model to distinguish between them. In this work, we investigate the
application of contrastive learning to the domain of medical image analysis.
Our findings reveal that MoCo v2, a state-of-the-art contrastive learning
method, encounters dimensional collapse when applied to medical images. This is
attributed to the high degree of inter-image similarity shared between the
medical images. To address this, we propose two key contributions: local
feature learning and feature decorrelation. Local feature learning improves the
ability of the model to focus on the local regions of the image, while feature
decorrelation removes the linear dependence among the features. Our
experimental findings demonstrate that our contributions significantly enhance
the model's performance in the downstream task of medical segmentation, both in
the linear evaluation and full fine-tuning settings. This work illustrates the
importance of effectively adapting SSL techniques to the characteristics of
medical imaging tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要