Knowledge-Embedded Transformer for 3-D Human Pose Estimation

Despite the significant advancements in deep learning, the task of estimating 3-D human pose and shape from a monocular RGB image remains challenging, primarily due to depth ambiguity. Regression approaches based on the SMPL model iteratively predict joint rotations, human shape, and camera parameters through a deep neural network. However, these approaches encounter difficulties in aligning predicted meshes with image pixels, attributed to cumulative errors arising from complex limb movements and occlusion. To address this challenge, we propose a left-right limb appearance consistency (LAC) module to enhance the limb reconstruction precision. Our approach utilizes a feature pyramid to correct model-predicted parameters based on mesh and image alignment status, progressively refining human meshes. To improve limb pose estimation, we incorporate prior knowledge of limb symmetry into the networks, employing a cross-attention mechanism that models spatial dependencies between left and right limbs. Unlike conventional regression-based methods that use weak perspective projection, we employ full perspective projection to obtain 2-D projection points across the entire image, subsequently revising the reprojection loss function. Furthermore, we introduce human global position information as supervision during network training to compensate for missing positional data due to cropping. This enables the network to accurately predict global rotation in the original camera coordinate system. Extensive experiments on widely used datasets demonstrate that our approach yields improvements compared with several state-of-the-art methods. The code is available at: https://github.com/XTU-PR-LAB/PAMA