Multimodal-Based Pedestrian Crossing Intention Recognition Method*

The ability of autonomous vehicles to accurately predict the future intention of pedestrians has become critical for ensuring road safety. However, accurately recognizing pedestrian behavioral intention remains a significant challenge. To address this challenge, we propose a novel multimodal-based pedestrian crossing intention recognition method that takes into account the pedestrian's abstraction and subjective mobility from the perspective of individual pedestrian behavior. Our proposed network architecture comprises three sub-networks, each dedicated to extracting specific features from different parts of the pedestrian. Specifically, one sub-network is designed to extract pedestrian skeleton features, another to capture head features, and a third to extract leg features. Additionally, a fusion network is incorporated to effectively combine these extracted features in an adaptive manner. The adaptive fusion process leverages the attention mechanism to learn fusion weights, dynamically adjusting the contribution of each feature channel based on their importance or relevance to the given task at hand. Overall, the proposed pipeline provides new state-of-the-art results on pedestrian intention recognition.