Offline Tracking with Object Permanence

To reduce the expensive labor costs of manually labeling autonomous driving datasets, an alternative is to automatically label the datasets using an offline perception system. However, objects might be temporarily occluded. Such occlusion scenarios in the datasets are common yet underexplored in offline auto labeling. In this work, we propose an offline tracking model that focuses on occluded object tracks. It leverages the concept of object permanence, which means objects continue to exist even if they are not observed anymore. The model contains three parts: a standard online tracker, a re-identification (Re-ID) module that associates tracklets before and after occlusion, and a track completion module that completes the fragmented tracks. The Re-ID module and the track completion module use the vectorized lane map as a prior to refine the tracking results with occlusion. The model can effectively recover the occluded object trajectories. It significantly improves the original online tracking result, demonstrating its potential to be applied in offline auto labeling as a useful plugin to improve tracking by recovering occlusions.

I. INTRODUCTION

Supervised deep learning-based models have achieved good performance in autonomous driving. However, it usually requires a huge amount of labeled data with high quality to train and tune such data-hungry models. An effective way is to auto label datasets, where labels can be automatically provided by a trained perception system. Waymo first proposed to auto label data offline to improve the quality of the generated labels [17]. In online tracking, the location of an object is inferred only from past and present sensor data. Online trackers are thus likely to produce false associations under severe occlusions. Offline multi-object tracking (MOT) is acausal and the position of an object can be inferred from past, present, and future sensor data. A consistent estimate of the scene can thus be optimized globally using the data not limited to a short moment in the past, enabling accurate object tracking even under severe occlusions. Based on global information, [25], [17], [8], [15] have developed offline auto labeling pipelines that generate accurate object trajectories in 3D space from LiDAR point cloud sequence data.