Stability Enhanced Hierarchical Reinforcement Learning for Autonomous Driving with Parameterized Trajectory Action

Reinforcement Learning (RL) has become a potential method for autonomous driving to adapt to complex driving environments with high flexibility. However, the popular RL paradigm directly outputting the vehicle control commands makes the future motion with fluctuation. To improve the driving behavior stability of RL method while ensuring the motion flexibility, this paper proposes a stability enhanced hierarchical reinforcement learning method based on parameterized trajectory action (RL-PTA). It offers feasible driving path in the long horizon and real-time control commands in the short horizon simultaneously. The RL agent actively contributes to path generation with discrete-continuous hybrid parameter actions, and the parameterized action space also ensures optimal consistency of the hybrid output. The experiment results show that the proposed method can generate flexible and stable lane-change driving behavior, thereby improving the efficiency and safety for autonomous driving.