PreGSU: A Generalized Traffic Scene Understanding Model for Autonomous Driving Based on Pretrained Graph Attention Network

Scene understanding, defined as learning, extraction, and representation of interactions among traffic elements, is one of the critical challenges toward high-level autonomous driving (AD). Current scene understanding methods mainly focus on a single concrete task, such as trajectory prediction and risk level evaluation. Although they perform well on specific metrics, the generalization ability is insufficient to adapt to the complexity of real-world traffic and the diversity of downstream demands. In this study, we propose PreGSU, a generalized pretrained scene understanding model based on a graph attention network to learn the universal interaction and reasoning of traffic scenes to support various downstream tasks. After feature engineering and subgraph module processing, all elements are embedded as nodes to form a dynamic weighted graph. Then, four graph attention layers are applied to learn the relationships among agents and lanes. In the pretraining phase, the understanding model is trained on two self-supervised tasks: virtual interaction force (VIF) modeling and masked roadmap modeling (MRM). Based on the artificial potential field theory, VIF modeling enables PreGSU to capture the agent-to-agent interactions, while MRM extracts agent-to-road connections. In the fine-tuning process, the pretrained parameters are loaded to derive detailed understanding outputs. We conduct validation experiments on three datasets and two downstream tasks, i.e., trajectory prediction in urban scenario and intention recognition in highway scenario, to verify the model’s generalization and understanding capabilities. Results show that compared with single-task-driven baselines, PreGSU achieves competitive performance on all datasets and downstream tasks, indicating its potential to be generalized to various scenes and targets. Ablation study shows the effectiveness of pretraining task design.