Abnormal Behavior Detection Method based on Spatio-temporal Dual-flow Network for Surveillance Videos

To address the issues that current surveillance videos abnormal behavior detection methods are affected by complex environments such as video blurring and long distances, resulting in low detection accuracy and slow speed, we propose an abnormal behavior detection method based on spatio-temporal dual-flow network for surveillance videos. Firstly, the input surveillance videos are sampled with different frame rates and a spatio-temporal dual-flow partial convolution network is constructed to extract spatial and motion information features from the spatial and temporal flow video sequences, respectively. Then, a cross-modal dual-attention fusion mechanism is introduced after each feature extraction of the dual-flow partial convolution network to enhance the exchange of feature information between spatial and temporal flows. Finally, the extracted motion and spatial features are fused to output detection results. Experiments show that our method reduces the number of floating-point operations and model parameters on the Kinetics-600 dataset compared to the MViT-L algorithm with guaranteed accuracy. Compared to lightweight networks such as X3D-XL, it improves the accuracy in Top-1 and Top-5 by 4.3% and 3.6%, respectively. The experiments prove that the method can reduce the model’s computational complexity while obtaining more accurate abnormal behavior detection results when the surveillance videos is blurred or the abnormal behavior occurs far away.