Tracking textured deformable objects using a finite-element mesh

This paper presents an algorithm for the estimation of the motion of textured objects undergoing nonrigid deformations over a sequence of images. An active mesh model, which is a finite-element deformable membrane, is introduced in order to achieve efficient representation of global and local deformations. The mesh is constructed using an adaptive triangulation procedure that places more triangles over high detail areas. Through robust least squares techniques and modal analysis, efficient estimation of global object deformations is achieved, based on a set of sparse displacement measurements. A local warping procedure is then applied to minimize the intensity matching error between subsequent images, and thus estimate local deformations. Among the major contributions of this paper are novel techniques developed to acquire knowledge of the object dynamics and structure directly from the image sequence, even in the absence of prior intelligence regarding the scene. Specifically, a coarse-to-fine estimation scheme is first developed, which adapts the model to locally deforming features. Subsequently, principal components modal analysis is used to accumulate knowledge of the object dynamics. This knowledge is finally exploited to constrain the object deformation. The problem of tracking the model over time is addressed, and a novel motion-compensated prediction approach is proposed to facilitate this. A novel method for the determination of the dynamical principal axes of deformation is developed. The experimental results demonstrate the efficiency and robustness of the proposed scheme, which has many potential applications in the areas of image coding, image analysis, and computer graphics.