Traffic Light and Uncertainty Aware Pedestrian Crossing Intention Prediction for Automated Vehicles

Predicting Vulnerable Road User (VRU) crossing intention is one of the major challenges in automated driving. Crossing intention prediction systems trained only on pedestrian features such as human pose key-points and bounding boxes underperform in certain situations like red/green light at traffic junctions, that are most obvious to humans which affects their crossing behavior. Moreover, such systems tend to be over-confident for out-of-distribution samples, therefore making them less reliable to be used by downstream tasks like sensor fusion and trajectory planning for automated vehicles. In this work, we demonstrate that the results of crossing intention prediction systems can be improved by incorporating the traffic light (TL) status as an additional input. Further, we make the model robust and interpretable by estimating uncertainty. Experiments on the PIE dataset show significant performance improvement of up to 5% on F1-score for three different state-of-the-art models for crossing intention prediction when using the traffic light status as an additional input. From these experiments we demonstrate the benefit of incorporating the TL-status into a crossing intention prediction model. Additionally, we show increased uncertainty values for out-of-distribution samples, therefore leading to more interpretable and more reliable predictions of crossing intention.