Level fusion analysis of recurrent audio and video neural network for violence detection in railway

This paper deals with the security improvement of passengers in public transport by automatically processing the audio and video streams of an embedded surveillance system. In this paper we analyse several levels of fusion of two deep audio and video recurrent network models for violent actions recognition. Each audio and video model is based on recent generic feature extractors proposed in the state-of-the-art to benefit of powerful feature representation capabilities. Each level of fusion is trained and evaluated on a new real-world audio-video surveillance streams recorded in a real train with scenes of violence played by actors. The obtained results confirm the interest in seeking to detect violence by jointly using audio and video signal and highlight the difficulty to define the optimal level of fusion.