The Benefits of Evaluating Tracker Performance Using Pixel-Wise Segmentations

For years, the ground truth data for evaluating object trackers consists of axis-aligned or oriented boxes. This greatly reduces the workload of labeling the datasets in the common benchmarks. Nevertheless, boxes are a very coarse approximation of an object and the approximation by a box has a large degree of ambiguity. Furthermore, tracking approaches that are not restricted to boxes cannot be evaluated within the benchmarks without adding a penalty to them. We present a simple extension to the VOT evaluation procedure that enables to include these approaches. Furthermore, we present upper bounds for trackers restricted to boxes. Moreover, we present a new measure that captures how well an approach can cope with scale changes without the need of frame-wise labels. We present a learning-based approach which helps to identify frames with heavy occlusion automatically. The framework is tested on the segmentations of the VOT2016 dataset.