Improving Video Anomaly Detection Performance with Patch-level Loss and Segmentation Map

With the development of surveillance video analysis and the increase of security requirements, video anomaly detection has attracted more and more attention. In the literature of video anomaly detection, since the ambiguity of anomaly and the lack of abnormal data, the researchers focus on the unsupervised methods that have the foreground-background imbalance problem. There is a method that employs attention-driven loss to alleviate this problem, but this method fails to be extended to multiple scenarios. In this paper, we propose a patch-level loss and utilize a segmentation map to overcome this problem in multiple scenarios. The patch-level loss is based on Multi-Scale Structural Similarity (MS_SSIM), making the network focus on the boundaries of the foreground in the training process. The segmentation map is derived from FastFCN, which changes the errors between predicted frames and tested frames in the test phase. Specifically, it strengthens the errors generated by the foreground and weakens those generated by the background. The proposed method increases the class spacing of normal and anomaly. Experimental results on several public datasets show that the proposed method consistently improves the performance of video anomaly detection.