Heuristic weakly supervised 3D human pose estimation

Estimating 3D human pose from 2D images in real world contexts remains a challenge, characterized by unique data constraints. Large general datasets of motion-captured 3D adult human poses paired with 2D images exist, but in many application settings, collection of further motion-captured data is impossible, precluding a straightforward fine-tuning approach to adaptation. We present a method for improving 3D pose estimation transfer learning to domains where there are only depth camera images available as supervision. Our heuristic weakly supervised 3D human pose (HW-HuP) estimation method learns partial pose priors from general 3D human pose datasets and employs weak supervision with depth data to guide learning in an optimization and regression cycle. We show that HW-HuP meaningfully improves upon state- of-the-art models in the adult in-bed setting, as well as on large scale public 3D human pose datasets, under comparable supervision conditions. Our model code and data are publicly available at https://github.com/ostadabbas/hw-hup. A significantly expanded version of this paper, with supplementary material, is available as a preprint on arXiv at https://arxiv.org/abs/2105.10996.