Moving Humans Detection Based on Multi-Modal Sensor Fusion

Moving object detection plays an important role in automated surveillance systems. However, it is challenging to detect moving objects robustly in a cluttered environment. In this paper, we propose an approach for detecting humans using multi-modal measurements. The approach is based on using Time-Delay Neural Network (TDNN) to fuse the audio and video data at the feature level for detecting the walker with multiple persons in the scene. The main contribution of this paper is the introduction of Time-Delay Neural Network in learning the relation between visual motion and step sounds of the walking person. Experimental results are presented.