In-bed Pressure Image-supported Diffusion for 3D Human Mesh Recovery

Reconstructing human dynamics in in-bed scenarios is important in dangerous behaviors detection, pressure sore prevention and sleep quality monitoring. Compared to RGB cameras, which are susceptible to obstruction and privacy concerns, pressure-sensitive bedsheets are gaining increasing attention for such tasks due to their non-invasive and privacy-preserving advantages. However, recovering a 3D human mesh from a single pressure image is challenging due to pressure information ambiguity, resulting in a high degree of uncertainty. Diffusion models, leveraging their powerful fitting capabilities and editing abilities, are promising candidates for addressing this challenge. Therefore, we propose a Pressure Image-supported Diffusion framework for Human Mesh Reconstruction (PIDHMR), which includes a direct regression of the human body model parameters and a follow-up optimization using information automatically detected from pressure images. For regression, we improve the existing diffusion model structure and training objectives to meet the task requirements for generating 3D human shapes from pressure images. For optimization, we examine human-related information that can be used to improve the predicted human mesh from pressure images (i.e., contact area, joints positions, and temporal consistency), and design optimization losses to utilize these information to improve the spatial position, posture, and motion smoothness separately. Ultimately, PIDHMR is evaluated on the public temporal pressure dataset, TIP, and achieves 72.83mm joint position errors, outperforming the state-of-the-art method PIMesh, which achieves 79.17mm.