Video object tracking with a sequential hierarchy of template deformations

We have developed a new contour-based tracking algorithm that uses a sequence of template deformations to model and track generic video objects. We organize the deformations into a hierarchy: globally affine deformations, piecewise (locally) affine deformations, and arbitrary smooth deformations (snakes). This design enables the algorithm to track objects whose pose and shape change in time compared to the template. If the object is not a rigid body, we model the temporal evolution of its shape by updating the entire template after each video frame; otherwise, we only update the pose of the object. Experimental results demonstrate that our method is able to track a variety of video objects, including those undergoing rapid changes. We quantitatively compare our algorithm with its constituent pieces (e.g., the snake algorithm) and show that the complete algorithm can track objects with moving parts for a longer duration than partial versions of the hierarchy. It could be benefited from a higher level algorithm to dynamically adjust the parameters and template deformations to improve the segmentation accuracy further. The hierarchical nature of this algorithm provides a framework that offers a modular approach for the design and enhancement of future object-tracking algorithms.