MDPose: Human Skeletal Motion Reconstruction Using WiFi Micro-Doppler Signatures

Motion tracking systems based on optical sensors typically suffer from poor lighting, occlusion, limited coverage, and may raise privacy concerns. Recently, radio-frequency (RF) based approaches using WiFi have emerged which offer low-cost ubiquitous sensing whilst preserving privacy. However, output range-Doppler or time-frequency spectrograms cannot represent human motion intuitively and usually requires further processing. In this study, we propose MDPose, a novel framework for human skeletal motion reconstruction based on WiFi micro-Doppler. MDPose provides an effective solution to represent human activity by reconstructing skeleton models with 17 key points, which can assist with the interpretation of conventional RF sensing outputs in a more understandable way. Specifically, MDPose is implemented over three sequential stages to address various challenges: First, a denoising algorithm is employed to remove any unwanted noise that may affect feature extraction and enhance weak Doppler measurements. Second, a convolutional neural network (CNN)-recurrent neural network (RNN) architecture is applied to learn temporal-spatial dependency from clean micro-Doppler and restore velocity information to key points under the supervision of the motion capture (Mocap) system. Finally, a pose optimisation mechanism based on learning optimisation vectors is employed to estimate the initial skeletal state and to eliminate additional errors. We have conducted comprehensive evaluations in a variety of environments using numerous subjects with a single receiver radar system to demonstrate the performance of MDPose, and report 29.4mm mean absolute error over key points positions on several common daily activities, which has performance comparable to that of state-of-the-art RF-based pose estimation systems.11For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) license to any Author Accepted Manuscript version arising.