A Deep Learning Dual-Stream Framework for Fall Detection

There is a need to improve fall detection systems to better discriminate falls from regular daily activities. Multi-sensor systems were proposed for this purpose. However, using wearable sensors may not be practical for elderly people. The use of surveillance cameras and advanced computer vision algorithms is thus an attractive solution. Indeed, raw RGB frames are rich in information including the human body shape and motions during activities. Yet, several disturbing features extracted from the raw frames may prevent distinguishing between normal activities and fall. To address this issue, we here leverage the human skeleton, and the binary history of motion during the activity. The former was analyzed with a Convolution Long Short Memory (ConvLSTM) to include spatial features during sequence learning. The latter was processed with a lightweight network for feature extraction. The outputs from both streams were merged to discriminate between fall and normal activities. Using the UP Fall dataset, our model detected falls with a single camera with a test accuracy of 100%, an improvement of 0.01% over current state-of-the-art methods.