A New Multimodal Video Detection Model and Dataset

In this work, we discuss the issue of multimodal detection of long and short video violations on Internet environment. We collect and annotate a dataset consisting of 6000 videos from different platforms, and offer a joint model of video frame sequences and text sequences to analyze video and text information for violation detection. We also do a comparison with the unimodal model. As far as we know, this is the first multimodal dataset proposed for the detection of violations involving “pornography and violence”. And the model adapts to the conversion of video unimodal data to multimodal data. Experiments found that although text unimodal recognition results are limited, the complementarity of the modalities in the multimodal case gives a large improvement in the results. We discuss the reasons for the results and open the codes and dataset in this domain for further study. The address is https://github.com/weijingxuan/multivideo.