Efficient Violence Recognition System using Spatio-Temporal Shift Multi-Scale Attention Model

One of the most challenging and crucial tasks in surveillance systems is detecting violent behavior from surveillance videos. Since surveillance videos are captured by a large number of surveillance cameras placed in various locations at any time, violence recognition is considered to be more difficult and crucial. For real-time surveillance videos, designing an efficient automated violence recognition system seems to be an imperative task that significantly reduces the time it takes for surveillance personnel to detect crime occurrences. The violence recognition approach has two main components, namely behavior description and behavior classification. Critical information for behavior description is obtained by extracting selectively prominent features from video data. Subsequently, the testing behavior is labeled as violent or non-violent with the help of a classifier trained using the extracted features. A novel Violence Behavior Recognition (VBR) system is introduced based on a deep learning approach, which includes the Spatio-Temporal Shift Module (STSM) to obtain multichannel features and the Multi-Scale Attention Module (MSAM) to focus on the most suitable feature components for violent behavior recognition. Salient regions for violence detection are derived from the boundaries of moving objects, calculated by measuring the Euclidean distance between two adjacent frames. The proposed method is evaluated through experiments on four violence datasets and demonstrates better performance when compared to state-of- the-art methods.