Deep Learning Driven Wireless Real-time Human Activity Recognition

Human activity recognition based on wireless sensing is advantageous at various features such as privacy preservation, but also very challenging due to the instability of wireless signals. This paper proposes a deep learning driven wireless human activity recognition solution based on Multiple-Input-Multiple-Output (MIMO) radar sensing. User activities are first sensed by a low-power Frequency-Modulated Continuous Wave (FMCW) MIMO radar. Then a sequence of 3D images are generated out of the reflected signal strength. Next, deep neural networks (DNNs) are designed to analyze the correlation among the sequential 3D images to recognize various types of human activities. This work has developed: 1) a large dataset containing over 1,500 training videos of six different types of indoor activities, 2) a customized deep learning video data-loader to select proper training data in each training epoch, 3) a deep recurrent neural network (RNN) model to recognize human activities based on radar imaging results. This solution has been extensively evaluated in a research lab room. The results show that the solution is able to generate wireless imaging frame-by-frame, and it can achieve over 86.7% accuracy in recognizing six different types of human activities.