PoinTS: mmWave Radar-based PointNet and Transformer Network for Human Skeleton Estimation in Privacy-Preserving Rehabilitation

Human skeleton estimation aims to predict 3D joint coordinates and skeletal structures from sensor data. Currently, mmWave-based radar human skeleton estimation has gained attention as it provides various advantages compared to RGB cameras, depth cameras, and wearable solutions. This paper proposes a novel deep learning architecture named PoinTS, which leverages the advantages of PointNet and Transformer encoder architectures in point cloud completion and sequence transduction, respectively, to provide highly accurate human skeleton estimation from radar point cloud data. This allows for privacy-preserving and markerless skeleton reconstruction. The model is evaluated on a publically available dataset whose usage is focused on assistive rehabilitation. The proposed method shows improved performance compared to three different state-of-the-art methods.