Cognitive Traffic Accident Anticipation

Traffic accident anticipation (TAA) in driving videos aims to provide early warning of potential accidents and support decision making in safe driving systems. Previous works typically focused on the spatial–temporal correlation of object-centric contexts but struggled to adapt to inherent long-tailed data distribution and severe environmental changes. In this article, we propose a cognitive TAA (Cog-TAA) method by leveraging the human-inspired cognition of driver fixations and textual scene descriptions based on visual observations to facilitate model training. Specifically, text descriptions offer dense semantic guidance for the primary context of traffic scenes, while driver attention directs focus to critical regions closely related to safe driving. Cog-TAA is formulated through an attentive text-to-vision shift fusion module, an attentive scene context transfer module, and a driver attention-guided accident anticipation module. We use the attention mechanism in these modules to discover crucial semantic cues for accident anticipation. To train Cog-TAA, we expand the existing self-collected DADA-2000 dataset (with annotated driver attention for each frame) by adding factual text descriptions for visual observations before accidents. Extensive experiments on DADA-2000 and the CCD dataset demonstrate Cog-TAA’s superiority compared to state-of-the-art approaches.