Improved Violence Detection in Video Analysis with a Hybrid CNN Approach Using 3D and 2D Convolutional Networks

Surveillance systems are essential to address the risk of criminal activities. It is crucial to detect such activities in real time to prompt an immediate police response. Currently, researchers are developing, training, and testing deep-learning models to detect these behaviors. In a violence detection system, capturing both spatial and temporal information is essential for accurately classifying violent activities. 3D Convolutional Neural Networks (CNNs) are particularly effective and common methods for extracting spatiotemporal features because they can model both spatial and temporal features in video sequences. However, 3D CNNs are known to be computationally intensive and require more training data compared to 2D CNNs. To address this problem, we propose a hybrid model by a fusion of 3D CNNs, a 2D depth-wise CNN, and a 2D CNN to classify violent activities. Employing a hybrid CNN not only reduces the complexity of the model, but it can also exhibit robust performance in a limited quantity of training samples. We exploit a 2D depth-wise CNN for spatial attention as it applies a single filter to each input channel individually without mixing information across channels. Therefore, we use it in the proposed model for spatial attention. Finally, the proposed approach is evaluated on four benchmark datasets: Hockey Fight, Surveillance Fight, Violent Flows, and Action Movies, achieving accuracy rates of 99.3%, 98.46%, 99.92%, and 98.2% respectively on each dataset.