Triple-Stream SWIN Transformer Based Encoder-Decoder for Image Forgery Localization

Due to the presence of so many image manipulating tools and technologies, the problem of image tampering has become widespread, resulting in a range of misleading and adverse consequences, and at the same time it has become very challenging to Law Enforcement Agencies. Image manipulation leaves behind both high level artifacts such as uneven contrast, and low level artifacts like pixel noise inconsistency, that create significant differences in image compression levels which can be analysed using the Error Level Analysis (ELA) technique. ELA, while very effective, is dependent on JPEG compression algorithm and can be influenced by image resizing and quality. This paper introduces a new three stream encoder decoder model using Swin transformer blocks to isolate manipulated parts in forged images. It make use of both high and low level features present on the tampered image to detect forgery artifacts and proposes to combine the error level information with forgery artifacts for precise localization. The first stream learns high-level image features using RGB values, while the second stream learns error level disparities utilizing ELA. The third stream uses low-level features using noise residuals after a high pass filter. The generated feature maps are combined and passed to the last sigmoid layer to get pixel probabilities. Thus, the feature mapping is realized in two ways - late fusion and middle fusion. The proposed model is trained on a synthetic dataset created using image segmentation datasets and tested on the CASIA v2 benchmark dataset achieving state-of-the-art performance of 96.2%.