CLIP-Enhanced Linear Alignment for AI-Generated Image Detection Using Latent Cross-Attention

With the rapid development of generative models, it has become increasingly challenging for humans to distinguish between AI-generated and real images, which may lead to the rampant spread of AI-generated misinformation. Therefore, detecting AI-generated images is a critical task to address this issue. In this work, we propose a two-stage method for Real/Fake detection and local detection of manipulated images. For Real/Fake detection, we are the first to propose rearranging the spatial features of large-scale pre-trained multimodal model encoders to achieve this goal. To this end, we introduce a linear alignment module in the first stage to fine-tune the pre-trained model. The fine-tuned model produces spatial distributions with clearer decision boundaries compared to other methods, demonstrating state-of-the-art performance in Real/Fake detection, achieving an average accuracy of 89.8%. Due to the lack of high-quality local detection data, we have created a dataset comprising real images, manipulated images, and corresponding masks (used to identify the manipulated regions) for research on local detection. For local detection, we innovatively introduce the rearranged spatial features from the first stage as auxiliary conditions into the second stage of local detection. We leverage the rearranged spatial features for local detection by employing cross-attention in the latent space as the backbone. Compared to other methods, our method can more accurately localize the manipulated regions in the source image, achieving state-of-the-art performance with an IoU of 61.5% and an F1 score of 71.2%.