Detection of Deepfake Content Representing Social Harmful Events for Image Forensic Applications using Machine Learning

Generally, with respect to the socially significant events, multimedia forensics plays a crucial role in ensuring this content is authentic, reliable, and useful for public awareness. Current forgery detection methods proposed in the literature doesn’t focuses in such scenes. This is because of the nonavailability of any socially significant event based forged dataset that implies negative sentiments. Depending on this phenomenon, the paper proposed a novel dataset named as “Socially Significant Event based Forged Image Dataset (SSE-FID)” that consist of Socially significant event based forged scenes where several sensitive candidate objects are spliced in the scenes that portrays negative sentiments in various aspects of the society. The manipulated images of the SSE-FID dataset have been created manually using traditional tool and automatically using AI generation tool. Till date, the proposed dataset contain 5020 images (i.e., 2435 authentic and 2585 manipulated images) depicting socially significant event based scenes. For making the proposed dataset more effective, the ground truth of spliced sensitive objects are annotated and provided in the form of bounding boxes and binary masks. Also, using this dataset, analysis of the encoder-decoder frameworks for pixel level localization of the spliced objects has been performed. In this paper, we find that CNN-based frameworks mostly works for within-database scenarios, when training and testing forgeries are generated by the same algorithm/tools and thus fail to generalize with respect to AI based forged socially significant scenes. Owing to the various data distributions produced by various manipulation techniques, the paper also proposes to analyze domain adaptation based strategy on the encoder-decoder frameworks for better generalization of AI generated manipulated images with respect to the traditional tool based manipulated images. Experimental results reveal the noticeable generalization of the CNN based frameworks for forgery localization in AI based manipulated scenes with an average F1Score of 0.55.