The explosive growth of generative AI has saturated the internet with AI-generated images, raising security concerns and increasing the need for reliable detection methods. The primary requirement for such detection is generalizability, typically achieved by training on numerous fake images from various models. However, practical limitations, such as closedsource models and restricted access, often result in limited training samples. Therefore, training a general detector with few-shot samples is essential for modern detection mechanisms. To address this challenge, we propose FAMSeC, a general AIgenerated image detection method based on LoRA-based Forgery Awareness Module and Semantic feature-guided Contrastive learning strategy. To effectively learn from limited samples and prevent overfitting, we developed a forgery awareness module (FAM) based on LoRA, maintaining the generalization of pretrained features. Additionally, to cooperate with FAM, we designed a semantic feature-guided contrastive learning strategy (SeC), making the FAM focus more on the differences between real/fake image than on the features of the samples themselves. Experiments show that FAMSeC outperforms state-of-the-art method, enhancing classification accuracy by 14.55% with just 0.56% of the training samples.

The rapid progress of the DeepFake technique has caused severe privacy problems. Thus protecting facial data against DeepFake becomes an urgent requirement. Face protection can be regarded as a bidirectional process: Face-out-detection (FOD) and Face-in-forensics (FIF). For FOD, the detectability should be satisfied when using the protected face to replace other faces. For FIF, traceability should be guaranteed when the protected face is replaced by others. For this, we propose a Bidirectional Facial-data Protection Framework (BiFPro) to protect face data comprehensively. This framework is composed of three main parts: Watermarking embedding, Face-out-detection (FOD) and Face-in-forensics (FIF). For the FOD case, we ensure the vulnerability of the original face by embedding fragile watermarking. Once the protected facial image is used to replace other faces, the watermarking information will be corrupted in the synthesized face images which can be used to detect the authenticity of the protected facial images. As for the FIF case, we guarantee the traceability of the protected face image by embedding robust watermarking, with which the fake faces can be traced with the reserved watermarking even after the face is swapped. Experimental results demonstrate that our proposed BiFPro could generate the watermarking which is fragile to FOD and at the same time robust to FIF with an average watermark extraction success rate reaching more than 95% when defending against the four advanced DeepFake techniques. Finally, we hope this work can encourage more initiative countermeasures against DeepFake.

With the rapid development in media generation technologies, the creation of DeepFake videos is within everyone's reach. As the widespread diffusion of DeepFakes can lead to severe consequences (e.g., defamation, fake news spreading, etc.), detecting DeepFakes is becoming a crucial task within the forensic community. However, most of the existing DeepFake detectors suffer from two issues: i) they are hardly explainable as they build upon black-box data-driven techniques rather than interpretable features; ii) they are often tailored to low-level texture features, failing to generalize on low-quality DeepFake videos. In this work we propose a video DeepFake detector that aims at solving these issues. The proposed detector relies on the fact that most DeepFake generators work on a frame-by-frame basis, thus breaking the temporal consistency of facial features across frames. In particular, we noticed that facial identity features tend to be less stable in time on DeepFake videos than original ones. We therefore propose a framework trained on time series of facial identity features. The use of high-level semantic features makes the detector interpretable and robust against low-quality DeepFake videos. Extensive experiments show that our method achieves outstanding performance on low-quality DeepFake video and obtains promising results on unseen dataset evaluation. The code is available at https://github.com/HongguLiu/Identity-Inconsistency-DeepFake-Detection

Deepfake video has been rapidly developed and attracted public concerns due to its potential wide applications, deepfake videos can be easily distinguished by DNN-based detection approaches. As the vulnerability of DNNs, the adversarial attack can be an effective way to deteriorate the ability of deepfake detection, but current adversarial attack techniques are commonly designed for individual images, which are easily perceived at the video level. To reveal the weakness of current attack methods, we first propose a robust detector utilizing the temporal consistency to discriminate between the clean and perturbed ones aiming at weakly adversarial deepfake videos, achieving maximum success rates of 100%. Then we propose a novel framework for generating high-quality adversarial deepfake videos which can fool deepfake detectors and evade the detection of adversarial perturbations simultaneously. Two pivotal techniques are utilized for improving the visual quality and the imperceptibility of adversarial perturbations: (i) Optical flow is adopted to restrict the temporal coherence of adversarial perturbations among frames; (ii) An adaptive distortion cost that can measure the complexity of each frame and help to keep the adversarial modification imperceptible. We demonstrate the effectiveness of our methods in disrupting representative DNN-based deepfake detectors. Extensive experiments are conducted to show the great improvement in coherence, visual quality, and imperceptibility of the adversarial deepfake videos. Furthermore, We hope that our adversarial deepfake generation framework can shed some light on the detection methods to fix their weakness. (c) 2022 Published by Elsevier B.V.

The widespread use of deep learning techniques for creating realistic synthetic media, commonly known as deepfakes, poses a significant threat to individuals, organizations, and society. As the malicious use of these data could lead to unpleasant situations, it is becoming crucial to distinguish between authentic and fake media. Nonetheless, though deepfake generation systems can create convincing images and audio, they may struggle to maintain consistency across different data modalities, such as producing a realistic video sequence where both visual frames and speech are fake and consistent one with the other. Moreover, these systems may not accurately reproduce semantic and timely accurate aspects. All these elements can be exploited to perform a robust detection of fake content. In this paper, we propose a novel approach for detecting deepfake video sequences by leveraging data multimodality. Our method extracts audio-visual features from the input video over time and analyzes them using time-aware neural networks. We exploit both the video and audio modalities to leverage the inconsistencies between and within them, enhancing the final detection performance. The peculiarity of the proposed method is that we never train on multimodal deepfake data, but on disjoint monomodal datasets which contain visual-only or audio-only deepfakes. This frees us from leveraging multimodal datasets during training, which is desirable given their lack in the literature. Moreover, at test time, it allows to evaluate the robustness of our proposed detector on unseen multimodal deepfakes. We test different fusion techniques between data modalities and investigate which one leads to more robust predictions by the developed detectors. Our results indicate that a multimodal approach is more effective than a monomodal one, even if trained on disjoint monomodal datasets.

近年来,得益于深度生成模型的发展,以Deepfake为代表的人脸深度伪造技术在互联网快速流行,受到了学术界和工业界的广泛重视。人脸深度伪造技术很容易被不法分子恶意使用,用来制作色情电影、虚假新闻,甚至被用于政要人物制造政治谣言等,这给国家安全和社会稳定都带来了极大的潜在威胁。为了降低深度伪造人脸视频所带来的负面影响,一系列深度伪造防御方法被提出。然而,由于数据集分布形式单一、评价标准不一致、主动性不足等问题,使得防御技术在走向实用的道路上仍有很长一段距离。本文将对深度伪造防御的相关研究工作进行科学系统的总结与归纳,并对现有技术的局限性做简要分析。最后,本文将探讨人脸深度伪造防御技术的潜在挑战与发展方向,为未来的研究工作提供借鉴。

With the rapid development of digital devices, the issue of how to transmit information among different devices with multimedia carriers has drawn much attention from the research community. This paper focuses on the important user scenario of “screen-to-camera information transmission”. Along this direction, image coding-based techniques have been shown to be the most popular and effective methods in the past decades. However, after careful study, we find that none of the existing methods can satisfy the four important properties simultaneously, i.e., high transparency , high embedding efficiency , strong transmission robustness and high adaptability to device types . This is mainly because these properties are contradictory with each other. In this paper, we thus propose a screen-to-camera image code dubbed “TERA” ( t ransparency, e fficiency, r obustness and a daptability), which makes it possible to circumvent the contradiction among the above four properties for the first time. Generally, TERA adopts the color decomposition principle to ensure the visual quality and the superposition-based scheme to ensure embedding efficiency. BCH-coding-based information arrangement and a powerful attention-guided information decoding network are further designed to guarantee the robustness and adaptability. Through extensive experiments, the superiority and broad applications of our method are demonstrated.

Recently almost all the mainstream deepfake detection methods use Convolutional Neural Networks (CNN) as their backbone. However, due to the overreliance on local texture information which is usually determined by forgery methods of training data, these CNN-based methods cannot generalize well to unseen data. To get out of the predicament of prior methods, in this paper, we propose a novel transformer-based framework to model both global and local information and analyze anomalies of face images. In particular, we design attention leading module, multi-forensics module and variant residual connections for deepfake detection, and leverage token-level contrast loss for more detailed supervision. Experiments on almost all popular public deepfake datasets demonstrate that our method achieves state-of-the-art performance in cross-dataset evaluation and comparable performance in intra-dataset evaluation.

近年来,得益于深度生成模型的发展,人脸的操控技术取得了巨大突破,以Deepfake为代表的人脸视频深度伪造技术在互联网快速流行,受到了学术界和工业界的广泛重视。这种深度伪造技术通过交换原始人脸和目标人脸的身份信息或编辑目标人脸的属性信息来合成虚假的人脸视频。人脸深度伪造技术激发了很多相关的娱乐应用,如使用面部替换技术将使用者的人脸替换到某段电影片段中,或使用表情重演技术来驱动某个著名人物的静态肖像等。但当前人脸深度伪造技术仍处于快速发展阶段,其生成的真实感和自然度仍有待进一步提升。另一方面,这类人脸深度伪造技术也很容易被不法分子恶意使用,用来制作色情电影、虚假新闻,甚至被用于政要人物来制造政治谣言等,这对国家安全与社会稳定都带来了极大的潜在威胁,因此伪造人脸视频的防御技术至关重要。为了降低深度伪造人脸视频所带来的负面影响,众多学者对伪造人脸视频的检测鉴别技术进行了深入研究,并从不同视角提出了一系列防御方法。然而由于数据集分布形式单一、评价标准不一致、主动性不足等问题,使得防御技术在走向实用的道路上仍有很长一段距离。事实上,人脸深度伪造与防御技术的研究仍旧处在发展期,其技术的内涵与外延正在快速的更新与迭代。本综述将对迄今为止的主要研究工作进行科学系统的总结与归纳,并对现有技术的局限性做简要分析。最后,本文将探讨人脸深度伪造与检测技术的潜在挑战与发展方向,为领域内未来的研究工作提供借鉴。

Deepfaketechniquecanproducerealisticmanipulationvideosincludingfull-facesynthesisandlocalregionforgery. General methods work well in detecting the former but are usually intractable in capturing local artifacts especially for lip forgery detection. In this paper, we focus on the lip forgery detection task. We first establish a robust mapping from audio to lip shapes. Then we classify the lip shapes of each video frame according to different spoken phonemes, enable the network in capturing the dissonances between lip shapes and phonemes in fake videos, increasing the interpretability. Each lip shape-phonemesetisusedtotrainasub-model,thosewithbetterdiscriminationwillbeselectedtoobtainanensembleclassificationmodel.Extensiveexperimentalresultsdemonstratethatourmethodoutperformsthemoststate-of-the-artmethodsonboth thepublicDFDCdatasetandaself-organizedlipforgerydataset.

Deepfake aims to swap a face of an image with someone else’s likeness in a reasonable manner. Existing methods usually perform deepfake frame by frame, thus ignoring video consistency and producing incoherent results. To address such a problem, we propose a novel framework Neural Identity Carrier (NICe), which learns identity transformation from an arbitrary face-swapping proxy via a U-Net. By modeling the incoherence between frames as noise, NICe naturally suppresses its disturbance and preserves primary identity information. Concretely, NICe inputs the original frame and learns transformation supervised by swapped pseudo labels. As the temporal incoherence has an uncertain or stochastic pattern, NICe can filter out such outliers and well maintain the target content by uncertainty prediction. With the predicted temporally stable appearance, NICe enhances its details by constraining 3D geometry consistency, making NICe learn fine-grained facial structure across the poses. In this way, NICe guarantees the temporal stableness of deepfake approaches and predicts detailed results against over-smoothness. Extensive experiments on benchmarks demonstrate that NICe significantly improves the quality of existing deepfake methods on video-level. Besides, data generated by our methods can benefit video-level deepfake detection methods.

近年来,得益于深度生成模型的发展,人脸的操控技术取得了巨大突破,以Deepfake为代表的人脸视频深度伪造技术在互联网快速流行,受到了学术界和工业界的广泛重视。这种深度伪造技术通过交换原始人脸和目标人脸的身份信息或编辑目标人脸的属性信息来合成虚假的人脸视频。人脸深度伪造技术激发了很多相关的娱乐应用,也很容易被不法分子恶意使用,对国家安全与社会稳定都带来了极大的潜在威胁,因此伪造人脸视频的防御技术至关重要。事实上,人脸深度伪造与防御技术的研究仍旧处在发展期,其技术的内涵与外延正在快速的更新与迭代。本综述将对迄今为止的主要研究工作进行科学系统的总结与分析,探讨人脸深度伪造与检测技术的潜在挑战与发展方向。

The remarkable success in face forgery techniques has received considerable attention in computer vision due to security concerns. We observe that up-sampling is a necessary step of most face forgery techniques, and cumulative up-sampling will result in obvious changes in the frequency domain, especially in the phase spectrum. According to the property of natural images, the phase spectrum preserves abundant frequency components that provide extra information and complement the loss of the amplitude spectrum. To this end, we present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery to improve the transferability, for face forgery detection. And we also theoretically analyze the validity of utilizing the phase spectrum. Moreover, we notice that local texture information is more crucial than high-level semantic information for the face forgery detection task. So we reduce the receptive fields by shallowing the network to suppress high-level features and focus on the local region. Extensive experiments show that SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.