Hybrid deep-learning framework for object-based forgery detection in video

Detection of object-based video forgery is receiving attention in recent years. However, until recently, dominant object-based forgery detectors are still based on hand-crafted features and their performances are not satisfactory. In this paper, we propose a novel hybrid deep-learning network which for the first time incorporates two-dimensional/three-dimensional convolutional neural network and recurrent neural network for object-based video forgery detection in videos with advanced encoding formats. Please note that the proposed framework is a full end-to-end data-driven solution. It is comprised of a specifically initialized three-dimensional convolutional layer which tries to mix up primitive intra-frame and inter-frame features, a two-dimensional CNN which tries to extract low-dimensional intra-frame features, and a four-layer bi-directional LSTM network which tries to catch high-level temporal features. Using this way, our proposed approach catches the intra-frame and inter-frame inherent properties of a target video with a united framework. The extensive experiments conducted on the largest object-based forged video database ever reported in the literature show that our hybrid framework achieves superior performance in forged video detection and forged segment localization. Moreover, the experiments conducted on datasets of videos with degraded quality demonstrated that our proposed framework is more robust in real-life scenarios.