Dual-Modal Human Pose Reconstruction using 3D UWB Radar Images and Point Clouds

This paper proposes a multimodal human pose re-construction method based on 3D ultra-wideband (UWB) radar images and point clouds, aiming to improve the accuracy of human pose estimation through the fusion of radar images and point cloud data. First, a UWB 3D imaging radar system is designed, which synchronously collects radar echo signals and optical images, con-structing a multimodal dataset covering various common actions and different human characteristics. Radar data processing in-cludes azimuth-range 2D imaging, target locking, local 3D imag-ing, discrete sampling, and maximum projection to generate point cloud data and projection images. Optical image processing uses mature methods to reconstruct 3D poses as pose labels for point clouds and projection images. To achieve multimodal data fusion, the UWB FusionPose network is designed, comprising an image feature extraction network, a point cloud feature extraction net-work, and a pose reconstruction network. The image feature ex-traction network is based on the ResNet-18 framework, while the point cloud feature extraction network adopts a pyramid struc-ture. After feature fusion, a multi-layer perceptron (MLP) is used to predict human pose information. Additionally, this paper ex-plores the impact of fusion parameters on network performance and verifies the effectiveness of the multimodal network through ablation experiments. Experimental results show that this method effectively utilizes radar point cloud data and projection image data to accurately reconstruct the 3D pose of human targets. This research not only provides a new human pose reconstruction tech-nique but also offers valuable references for the future develop-ment of radar imaging technology and multimodal data fusion methods.