GPU-Accelerated KLT Tracking with Monte-Carlo-Based Feature Reselection

Many computer vision methods rely on frame registration information obtained with algorithms such as the Kanade-Lucas-Tomasi (KLT) feature tracker, which is known for its excellent performance in that area. Various research groups proposed methods to extend its performance, both in terms of execution time and stability. Recent research has shown that current graphics processing units (GPUs) have proven to be very efficient SIMD parallel processing architectures that can be used to speed up the execution time of the KLT algorithm by an order of a magnitude compared with an ordinary CPU implementation and thus making it suitable for real-time applications on commodity hardware. Previous publications demonstrated the use of the GPU for the tracking and image processing part of the KLT algorithm. One essential, but computationally demanding step of the KLT algorithm is the feature selection step. It injects fresh feature points into the existing point set and thus enables the KLT to continuously track points throughout an image sequence. Those sequences can contain rapid movements or they may be of low quality. In such situations, features diminish rapidly and the step must be performed potentially for every frame. Thus, the performance of the otherwise efficient GPU implementation declines substantially, as this step includes a sorting operation on the sparse feature map, which is difficult to implement efficiently on the GPU. We use the KLT algorithm to calculate a stable frame registration with high accuracy using the feature point set. We found that maintaining a well distributed feature set through frequent injections of points is an essential requirement. In this paper we will demonstrate an alternative feature reselection method that can be efficiently implemented on the GPU. It is a Monte-Carlo-based approximation of the original method and leads to very good tracking results with just a fraction of the computational cost.