FBI-Net: Frequency-Based Image Forgery Localization via Multitask Learning With Self-Attention

Image forgery is easily manufactured for illegal acts such as spreading misleading information, which can have unfortunate consequences for society. In this work, we propose a Discrete Cosine Transformation (DCT) based multi-task learning network named FBI-Net, for forgery localization. Our proposed network adopts a fully convolutional encoder-decoder architecture, consisting of three encoders sharing parameters, a bridge attention module, and two output streams in the decoder. The encoder takes three inputs: RGB images and high-/low- DCT-filtered images. High-frequency components help learn object characteristics that improve CNN accuracy; low-frequency components are essential frequency information to keep most of the energy found in the typical DCT. Subsequently, Dilated Frequency Self-Attention Module, DFSAM in the bridge layer, is incorporated into the network to recalibrate the fused features and enhance the representation. Finally, in the decoder stage, region and edge information of the label are learned through multi-task learning to provide more extensive supervision for forged region localization; the edge stream will give a deeper understanding of features between forged and authentic images and help learn how to predict exquisite representations in images. Simultaneously, the auxiliary features from the pre-trained segmentation model are fused to separate the segmented background and objects, drawing the segmentation result of the dense region obtained. Extensive experiments show that our proposed FBI-Net outperforms existing forgery localization methods on six benchmark splicing and copy-move image datasets, CASIA TIDE v 1.0, CASIA TIDE v 2.0, Carvalho, Columbia, Coverage, IMD2020, achieving the best performance in an average of IoU of 70.99% and F1-score of 76.98% which is 9.79%, 9.82% higher than the previous method, respectively.