Qualitative Tracking Performance Evaluation without Ground-Truth

We present a qualitative tracking performance evaluation algorithm without ground-truth, where several representative frames are automatically selected and visualized in a principled way. Although tracking algorithms are typically evaluated by quantitative scores based on predefined measures, qualitative evaluation is also useful especially when the ground-truth of a target state is unavailable or unreliable. However, there is no prior study on how to present frames for better qualitative evaluation of tracking algorithms. Motivated by this fact, we propose an unbiased frame selection technique, where salient and unique features in tracking results are captured effectively. Our method identifies a set of representative frames by 1) analyzing the sequence structure using manifold learning, and 2) selecting frames by formulating the task as a facility location problem. By presenting the manifold and the selected frames, one can understand the sequence structure as well as the characteristics of tracking results. The effectiveness of our method is illustrated with single and multiple tracking results for sequences without ground-truth.