Confidence-Based Multibody Kinematics Optimization for Markerless Motion Capture: Evaluation on Synthetic Data

Markerless motion capture methods open the way for lighter motion analysis setups, but efforts are still needed to improve the obtained 3D kinematics. Using videos, point estimation software generate 2D confidence heatmaps. Only the position of the pixel with maximum confidence is usually used for triangulation, which neglects other possible information in the camera plane. We present and evaluate a confidence-based multibody kinematics optimization (MKO) method, which maximizes the summed 3D confidence of the model-derived points. This summed 3D confidence is obtained in a continuous and differentiable form by combining information from 2D confidence heatmaps of the surrounding cameras that were parameterized as 2D Gaussian functions. This confidence-based MKO method was evaluated using synthetic data. Typical noise of point estimation software was added to reference data in order to generate the synthetic data. Confidence-based and classical distance-based MKO methods were applied to the synthetic data. Results (joint angles and 3D point positions) were compared to those obtained with a distance-based MKO applied to the reference data. It showed a better agreement for the confidence-based MKO method and robustness to missing data, suggesting that the confidence-based MKO method performs well.