Learning-to-Count by Learning-to-Rank

Object counting methods rely on density maps, which are heatmaps produced by placing Gaussian density over object locations. However, density maps are expensive to collect. To reduce the annotation burden, we propose a form of weak supervision that only requires object-based pairwise image rankings. These annotations can be collected rapidly with a single click per image pair and supply a weak signal for object quantity. However, a model learn to fit spurious patterns that satisfy the ranking constraint but do not rely on the objects. To encourage the network to solve the ranking constraints by localizing objects, we propose adversarial density map estimation. This method regularizes a ranking network's intermediate feature representation such that it corresponds to a plausible density map. We demonstrate the effectiveness of our method on several benchmark object counting datasets, and show results with a performance that approaches that of fully-supervised methods using data that can be collected with a fraction of the annotation burden. We release code for reproducibility: github.com/sfu-mial/Rank2Count