Target-Oriented Environmental Context Modeling for Pedestrian Risky Behavior Detection

In autonomous driving systems, detecting pedestrian risky behavior is crucial for ensuring the safety of human-vehicle interactions. The behavior of pedestrians is not only determined by their individual actions but also influenced by their interactions with the surrounding environment. Considering the impact of environmental context on pedestrian behavior, recent studies have proposed various methods for extracting contextual information. However, efficiently modeling the environmental contextual features of pedestrian behavior remains a challenge. In this paper, we propose a target-oriented environmental context modeling method that accounts for the role of target-specific features in constructing context features, achieving target-adaptive context feature extraction. Our solution, called State DEtection TRansformer (SDETR), provides an end-to-end framework for risky pedestrian behavior detection. First, we devise a Dual-level Feature Encoder that effectively decouples high-level target semantic and low-level environmental texture. Specifically, the texture encoding enables label-free environmental feature extraction. Then, we develop a Object-Environment Perception Decoder that flexibly decodes cross-feature domain contextual features using object features. Finally, a Feature Fusion Head is employed to merge object features with environmental state features for the detection output. Experiments demonstrate the outstanding performance of SDETR in two typical risky behavior detection tasks (crossing detection and intrusion detection). We report a new record of 87.3% accuracy on the JAAD dataset and 76.3% accuracy on the Cityintrusion dataset, which significantly outperforms all previously published results. Note to Practitioners—The motivation of this paper is to detect risky pedestrian behavior in traffic scenes, with a particular focus on the construction of pedestrian environmental context. Existing methods for extracting environmental context typically involve complex feature extraction or label descriptions of the background, followed by a mechanistic establishment of the interplay between pedestrians and their surroundings. This paper proposes a flexible and cost-effective method for modeling environmental context. We employ an attention mechanism to autonomously decode cross-feature domain contextual state features, using target features as query features. Moreover, we employ dual-level feature extraction methods for targets and backgrounds (target semantics and background textures), significantly reducing the labeling cost for environmental description. Preliminary experiments suggest that this method is feasible in the detection of two risky pedestrian behaviors: pedestrian crossing and pedestrian intrusion. However, it has not yet been extended to other pedestrian dangerous behavior tasks. In future research, we intend to delve deeper into the recognition of unlabeled pedestrian abnormal and risky behaviors, expanding our research beyond the current scope.