Multi-Scale Residual Encoder with Cascaded Attention for 3D Human Model Generation

By parameterizing a model of the human body, model-based methods in 3D pose and shape estimation reconstruct a comprehensive 3D mesh. However, the process of learning abstract parameters is highly nonlinear, and issues related to alignment and challenges in parameter learning can result in subpar model performance. Previous research integrates 3D keypoint estimation methods with deep convolutional neural networks (CNNs) and volumetric representations to attain precise localization accuracy at the pixel level. Nonetheless, CNN networks encounter issues such as feature redundancy and insufficient effective depth, which pose challenges in generating accurate 3D keypoints in complex scenarios and ultimately lead to diminished fidelity of the body mesh. This paper introduces a novel feature extraction network called MSRE (Multi-Scale Residual Encoder) to tackle these challenges. MSRE preserves additional low-dimensional spatial and pixel information, facilitating the interaction between local and global features in the network for accurate 3D keypoint determination and parameter learning. Furthermore, after generating the 3D volumetric heatmap through deconvolution, a cascaded attention mechanism is incorporated. The mechanism effectively mitigates the impact of artifacts during the upsampling process and places emphasis on capturing additional spatial positional information. The experimental results validate the high accuracy and reconstruction performance of our proposed network model, which achieves global optimality. Remarkable enhancements are observed across the 3DPW [1] and Human3.6M [2] datasets.