Future Frame Prediction Network for Human Fall Detection in Surveillance Videos

Video fall detection is one of the most significant challenges in computer vision domain, and it usually involves the recognition of events that do not conform to expected falls. Recently, a majority of unsupervised models are popular to address the issues that call for substantial manual labeled training data in supervised learning. However, almost all existing unsupervised methods usually minimize reconstruction errors, which may lead to insufficient reconstruction errors between fall and nonfall video frames because of the powerful representation ability of the neural network. In this article, we propose a novel efficient fall detection method based on future frame prediction. Specifically, attention U-Net with flexible global aggregation blocks that can achieve better performance is regarded as a frame prediction network, achieving that several video frames predict the next future frame. In the training phase, commonly used appearance constraints on intensity and gradient and motion constraint are combined to further generate higher quality frames. Such constraints promote the performance of the prediction network, which can enlarge the difference between the predicted fall frame and the real fall frame. In the testing phase, the fall score based on the error between the predicted frame and the real frame can be computed to distinguish the fall event. Exhaustive experiments have been conducted on the UR fall dataset, multiple cameras fall dataset (MCFD), and high-quality fall simulation dataset, and the results verify the effectiveness of the proposed method and outperform other existing state-of-the-art methods.