Ranking of Visual Trackers Using Robust Error Norms

Object trackers are typically ranked by the average of averages, that is, a performance measure averaged over all frames of a video and then averaged over the entire dataset. The average is not a robust estimator. We propose to rank trackers based on robust error norms: we divide the performances of a set of trackers for a video, sorted from best to worst, into outliers (edge trackers) and inliers (trackers with similar performances); we propose an edge-stopping function that assigns the highest score to the highest-performance (top) tracker and scores other trackers accordingly. Our edge-stopping function stops at edge trackers (outliers) using a robust scale defined using the difference (error) between the performances of the top tracker and neighboring trackers. Our method is not a new performance measure but an approach to rank trackers robustly and systematically. We test our methods using five video datasets and 20 trackers. We show that the proposed score is more robust and representative of a tracker’s performance than the widely-used average of averages.