Hierarchical Reinforcement Learning with Successor Representation for Intelligent Vehicle Collision Avoidance of Dynamic Pedestrian

In urban environments, intersection scenarios with dynamic pedestrians are challenging for intelligent vehicles (IV). To ensure safety and efficiency, it is crucial to develop an effective decision-making and control system that can avoid collision with other pedestrians. Considering pedestrian trajectory prediction is beneficial for collision avoidance systems. However, current collision avoidance methods mostly use short-time trajectory prediction results. The exploration and utilization of long-time trajectory prediction and behavior pattern of pedestrians are inadequate. to solve this problem, we propose a decision-making and control system based on pedestrian risk prediction. It combines the idea of successor representation (SR) with hierarchical reinforcement learning (HRL). It builds predictive risk map (PRM) based on SR, and implement decision-making and control by selecting decision primitives (DPs) to complete driving tasks. The proposed method is tested with dynamic pedestrians in intersection scenarios based on the CARLA simulator. The pedestrian trajectories in simulation are built from stochastic data as well as real-world data to reflect the dynamics and uncertainty of real pedestrians. After online learning and testing, the proposed method is proved to have the best performance compared to baseline methods.