HDD4DBP: A Large-Scale Multi-Modal Benchmark on Driving Behavior Prediction

Driving behavior prediction (DBP) plays an important role in autonomous driving. Accurately anticipating driving behaviors (e.g., left turn, right turn) of ego-vehicles 3–5 seconds before actual occurrences of maneuvers can help AI pilots planning safer trajectories or informing possible dangers to drivers. However, current DBP benchmark datasets, e.g., the Brain4Cars, are restricted by the limited amount of data samples and small number of behavior categories. Thus, there is an urgent need for creating a new larger-scale benchmark to help boost the studies of DBP. In this work, based on the annotations in the HRI Driving Dataset (HDD), we extract 5000+ clips from untrimmed multi-modal data sequences to form a new dataset, termed the HDD4DBP, as a convincing large-scale testbed for evaluating various DPB methods. Furthermore, inspired by the success of transformers on modeling long range dependence in sequences, we build a strong baseline with the vision transformer (ViT) backbone for predicting driving behaviors. Compared to previous representative baselines, a large margin performance gain can be achieved by our strong baseline on the HDD4DBP. Moreover, simply fine-tuning the pre-trained strong baseline can obtain the state-of-the-art performance on the Brain4Cars dataset, thereby futher confirming the benefits of the HDD4DBP. The dataset and source code will be released. Find data and code at https://github.com/HDD4DBP/MTL.