Visibility-Aware Human Mesh Recovery via Balancing Dense Correspondence and Probability Model

Reconstructing the human body mesh often faces challenges like self-occlusion, object occlusion, and interference from other people. However, focusing on the model's robustness in scenarios of occlusion leads to a compromise in the accuracy of estimating non-occluded humans. Striking the right balance is a research question worth exploring. In this study, we introduce the Visibility-aware Human Mesh Recovery (VisHMR) method, which dynamically adapts to different scenarios based on the visibility of different body parts. It enhances robustness in estimating occluded areas of the human body, while also accurately regressing the visible parts. Specifically, VisHMR introduces a dual-branch network that models and estimates occluded and non-occluded regions separately. In visible regions, we establish dense correspondence between the image and rotation in UV space. In occluded regions, we utilize a Normalizing Flow model to hypothesize the most probable poses. Besides, a fusion module is designed to integrate the results of the dual-branch network, yielding a precise yet robust rotation parameter. Extensive experiments reveal that VisHMR significantly outperforms previous state-of-the-art methods in dealing with occlusions, and performs comparably to the best methods on standard non-occluded benchmarks.