Exploring Contrastive Learning and CLIP for Improving Image Clustering

In this work, we proposed an advanced deep clustering approach that leverages a pre-trained image encoder from CLIP to enhance image clustering by refining the learned feature representations. Specifically, this approach first designs a consistency loss mechanism to ensure the alignment of pseudo-labels generated by the clustering head with feature representations derived from the instance head. This mechanism facilitates the acquisition of more reliable and coherent feature representations. Second, in lieu of conventional strong data augmentation techniques, this approach employs a random masking strategy to enhance the diversity of the image datasets, enabling the model to prioritize finer image details and potentially improve its generalization ability. The efficacy and superior performance of this framework have been demonstrated through extensive experimentation across six public image datasets: STL-10, CIFAR-10, CIFAR-100, ImageNet-Dog, ImageNet-10 and Tiny-ImageNet. Notably, the proposed method achieves impressive results on the CIFAR-100 dataset, surpassing existing techniques by up to 11% in Normalized Mutual Information (NMI). It records scores of 0.562 for Accuracy, 0.586 for NMI, and 0.413 for Adjusted Rand Index. Ablation studies further underscore the individual contributions of the feature model, consistency loss, and random masking components to the framework's overall performance. The code can be available at https://github.com/MMengjuan-Li/Contrastive-Learning-and-CLIP-for-Clustering.