Multi-Sensor Data Annotation Using Sequence-based Active Learning

Neural Networks are the state-of-the-art technology for environmental perception in applications such as autonomous driving. However, they require a large amount of training data in order to perform well, making the selection and annotation of sensor data a time-consuming and expensive task. Active learning is a promising approach to reduce the required amount of training data by selecting samples for annotation that are expected to improve the neural network the most. In this work, we propose a sequence-based active learning approach that selects sequences of consecutive frames instead of individual images. This allows to evaluate tracking algorithms and to reduce the annotation effort by interpolating labels between frames. Our approach is compared to a random sampling strategy as baseline. Over 15 iterations, both approaches select 1000 additional images for training in each iteration. The performance of the neural network trained on the data selected by our sequence-based active learning approach is compared to the performance of the network trained on the data select by the baseline approach. The results show that sequence-based active learning can reduce the required amount of training data by up to 25% while reaching a similar performance. Furthermore, sequence-based active learning can improve the neural network's overall performance by 2 % compared to a random sampling strategy. In this work, the proposed method was evaluated with a new dataset consisting of 15 scenes in railway environments. The dataset has 45,888 frames in total, 14,513 frames contain persons on railway stations or close to tracks.