A biomechanical model for image-based estimation of 3D face deformations

To reproduce a face motion from an image sequence, natural motion parameters provide a semantical and efficient way of representation. State-of-the art techniques for 3D face motion estimation employ a limited set of predefined key-shapes of face structure, and thereby restrict the possible face motion which can cause distortions. We propose a new approach in which such distortions are avoided by augmenting a 3D structural surface face model with a physical motion model originating from continuum mechanics. Implementation with a displacement-based FEM does not only describe, but also explain the motion of the face's skin tissue caused by the muscle force parameterized actuations. The correct usage of the motion model and the mapping of the 3D scene flow to the 2D optical flow, allow posing the 3D deformation estimation as an inverse problem, for which a solution has been obtained using numerical solvers.