F2Trans: High-Frequency Fine-Grained Transformer for Face Forgery Detection

In recent years, face forgery detectors have aroused great interest and achieved impressive performance, but they are still struggling with generalization and robustness. In this work, we explore taking full advantage of the fine-grained forgery traces in both spatial and frequency domains to alleviate this issue. Specifically, we propose a novel High-Frequency Fine-Grained Transformer (F2Trans) network which contains two important components, namely Central Difference Attention (CDA) and High-frequency Wavelet Sampler (HWS). The premier CDA module is capable of capturing invariant fine-grained manipulation patterns by aggregating both pixel-level intensity and gradient information of the query to generate key and value pairs. Subsequently, the proposed HWS discards the low-frequency components of wavelet transformation and hierarchically explores high-frequency forgery cues of feature maps, which prevents model confusion caused by low-frequency components and pays attention to local frequency information. In addition, HWS can be employed as a special pooling layer for the F2Trans architecture to produce hierarchical feature representations in the spatial-frequency domain. Extensive experiments on multiple popular benchmarks demonstrate the generalization and robustness of the specially designed F2Trans framework is well-tailored for face forgery detection when confronting the cross-dataset, cross-manipulation, and unseen perturbations.

I. Introduction

With the rapid development of deepfake techniques [4], [6], [7], various algorithms (e.g., NeuralTextures [3] and Face2Face [5]) have been proposed to generate realistic face images and videos, which are indistinguishable to human eyes or face recognition system [8], [9]. These forged media may be abused for unethical and malicious uses, e.g., spreading political propaganda and creating fake news, which poses huge threats to security. Under this background, face forgery detection came into being and has attracted more and more attention recently.