Pedestrian Crossing Prediction With Pathwise Feature Fusion and Stacked Gate Recurrent Unit

Real-time pedestrian crossing prediction is an essential prerequisite for automated driving systems, especially for urban driving. Vision-based deep neural network models have been investigated to solve the problem. However, certain challenges still need to be addressed, such as the optimal strategy for fusing different features from heterogeneous sensors. In this letter, a novel network architecture is proposed to fuse different spatial and temporal features for pedestrian crossing prediction. The network is divided into the vision and nonvision paths for corresponding information extraction. Meanwhile, we introduce the pathwise fusion block to exchange information between different paths for feature reuse. With the help of attention mechanisms and a stacked gated recurrent neural network, vision and nonvision features are fused in an optimal way. The effectiveness of the proposed method is evaluated on the PIE dataset, it outperforms the state-of-the-art benchmarks in terms of accuracy and precision.