Approximate 2D-3D Shape Matching for Interactive Applications

Matching a 2D contour to a non-rigidly deformed 3D mesh is a challenging problem due to ambiguities arising from dimensionality differences. In the past, product graph based methods were only able to either produce fast but noisy solutions, or smooth but slow solutions (the latter enabled by higher-order costs computed in the conjugate product graph). In this work, we propose an approximation of these higher-order costs so that they can be computed in the ordinary product graph. This leads to an efficient algorithm for high-quality 2D-3D shape matching and enables novel applications, like an interactive user interface which allows to refine the solution gradually. We show theoretically that our method is efficient, and we experimentally validate that the accuracy gap of our approximation to the optimum is small in practice. Our code is available.¹¹https://github.com/christophpetzsch/sm-2D3D-approx