Unscented KLT: nonlinear feature and uncertainty tracking

Accurate feature tracking is the foundation of several high level tasks, such as 3D reconstruction and motion analysis. Although there are many feature tracking algorithms, most of them do not maintain information about the error of the data being tracked. In this paper, we propose a new generic framework that uses the scaled unscented transform (SUT) to augment arbitrary feature tracking algorithms, by introducing Gaussian random variables (GRV) for the representation of features' locations uncertainties. Here, we apply the framework to the well-understood Kanade-Lucas-Tomasi (KLT) feature tracker, giving birth to what we call unscented KLT (UKLT). It tracks probabilistic confidences and better rejects errors, all on-line, and leads to more robust computer vision applications. We also validate the experiments with a bundle adjustment procedure, using real and synthetic sequences