An End-to-End Two-Branch Network Towards Robust Video Fingerprinting

With the increasing number of edited videos, many robust video fingerprinting schemes have been proposed to solve the problem of video content authentication. However, most of them either deal with the temporal and spatial features symmetrically or insufficiently consider the temporal information. In this work, an end-to-end two-branch network toward robust video fingerprinting (RVFNet) is proposed, where the two branches focus on the temporal and spatial information, respectively. The temporal branch aims to comprehensively capture complex motion patterns by combining subtle motion changes with the overall motion trend. The spatial branch exploits the pixel-level information obtained by multiple receptive fields while preserving significant structural features. Deep metric learning is employed in the training process, and we adopt hard triplet loss to constrain the generation of fingerprints. Furthermore, we construct a large-scale and complex dataset for the robust video fingerprinting task based on multiple video content-preserving manipulations in actual scenarios. The size of our dataset exceeds most datasets adopted in the current robust video fingerprinting schemes. Based on the proposed dataset, experimental results demonstrate that our scheme achieves outstanding performance improvements compared with the state of the art.