Consistent Learning of Multiple Paths Using Frequency-Aware Clues for Face Forgery Detection

Deepfake technology, in particular, has introduced significant cybersecurity issues in the realm of online media. This work realizes a four-path model based on Xception, leveraging augmented RGB images and inversed DCT images from five frequency bands with learnable masks. Two frequency-aware attention modules are inserted in the middle blocks of Xception, merging two dual paths to streamline the model to two paths. The consistent learning is applied to the corresponding blocks after attention in the two paths, for discovering the least similar one. The holistic loss comprises the consistency loss of the block outputs with the least similarity, and the cross-entropy of the classification result. Our model, when evaluated on the face forensics++ dataset, achieves exceptional Area Under Curve (AUC) values of 100.00%, 99.71%, and 90.75% at the original resolution (raw), high resolution (c23), and low resolution (c40), respectively. Notably, at raw and c23, our model outperforms conventional ones, showcasing superior performance and robustness in face forgery detection.