Sequential and Patch Analyses for Object Removal Video Forgery Detection and Localization

In recent years, video surveillance has become essential for security applications used to monitor many organizations and locations, and it is therefore important to ensure the reliability of these surveillance videos. Unfortunately, surveillance videos can be forged with little effort by deleting an object from a video scene while leaving no visible traces. A fundamental challenge in video security is to determine whether or not an object has been removed from a video. This task is particularly challenging due to the lack of ground truth bases that can be used to verify the originality and integrity of video contents. In this paper, we propose a novel approach based on sequential and patch analyses to detect object removal forgery and to localize forged regions in videos. Sequential analysis is performed by modeling video sequences as stochastic processes, where changes in the parameters of these processes are used to detect a video forgery. Patch analysis is performed by modeling video sequences as a mixture model of normal and anomalous patches, with the aim to separate these patches by identifying the distribution of each patch. We localize forged regions by visualizing the movement of removed objects using anomalous patches. We conduct our experiments at both pixel and video levels to determine the effectiveness and efficiency of our approach to detection of video forgery. The experimental results show that our approach achieves excellent detection performance with low-computational complexity and leads to robust results for compressed and low-resolution videos.