Connected vibrations: a modal analysis approach for non-rigid motion tracking

A new framework is presented in this paper for tracking non-rigid motion in image sequences. This method models non-rigid motions as the deformations of connected 2D membrane patches. Local smoothness constraints for each patch are the low-frequency vibration modes obtained from modal analysis. These constraints are incorporated with the template matching algorithm into a local motion estimator. To maintain the global topological structure, different patches are connected together by hinges, which introduce additional constraints in the tracking process. The resulted over-constrained linear system is solved efficiently using a least square estimator. To improve the robustness of the tracker, stochastic filtering technique is employed to take advantage of the temporal continuity of the deformations. Promising results on both natural and synthetic facial motion sequences are demonstrated in this paper.