Whole-body contact force sensing from motion capture

Using motion capture only, we challenge in this paper the estimation of the contact forces-backed with ground-truth sensing, in human whole-body interaction with the environment. Our method is novel and makes it possible to get rid of cumbersome force sensors in monitoring multi-contact motion and its interaction force data. This problem is very difficult. Indeed, while a given force distribution uniquely determines the resulting kinematics, the converse is generally not true in multi-contact. In such scenarios, physics-based optimization alone may only capture force distributions that are physically compatible with a given motion rather than the actual forces being applied. We address this indeterminacy by collecting a large-scale dataset on whole-body motion and contact forces humans apply in multi-contact scenarios. We then train recurrent neural networks on real human force distribution patterns and complement them with a second-order cone program ensuring the physical validity of the predictions. Extensive validation on challenging dynamic and multi-contact scenarios shows that the method we propose can outperform physical force sensing both in terms of accuracy and usability.