QDNet: Query-Denoising Network for Visual Traffic Knowledge Graph Generation

Traffic scene perception underpins essential tasks like map construction and route planning in modern intelligent transportation systems, thus receiving extensive attention. However, existing methods tend to concentrate solely on specific elements, lacking a comprehensive understanding of various traffic scenes. This paper addresses the Visual Traffic Knowledge Graph Generation (VTKGG) task, aiming to extract and represent traffic information from various elements in the traffic scene image as a knowledge graph. To achieve this, we propose Query-Denoising Network (QDNet) to integrate multiple subtasks through different types of queries in an end-to-end manner. These queries facilitate information communication between different modules, streamlining the generation of visual traffic knowledge graphs by eliminating cumbersome intermediate steps. Considering the challenges in optimizing such a cascaded multi-task model, we incorporate the query-denoising method into the training process of QDNet. By introducing the noised query, enhancing the internal noise of the model, and forcing the model to recover the ground truth, our approach achieves accurate results. This strategy improves the robustness and performance of our model. We conduct extensive ablation and comparative experiments to demonstrate the superiority and effectiveness of our framework and strategy, and experiments on a similar task Panoptic Scene Graph Generation also demonstrate its superiority.