An Explainable and Robust Motion Planning and Control Approach for Autonomous Vehicle On-Ramping Merging Task Using Deep Reinforcement Learning

Reinforcement learning (RL) has the capability to discover optimal interactions with the surrounding environment, with the advantage that nearly all required computations can be performed offline. Nevertheless, the lack of explainability for RL-based solutions may prevent their large-scale application in industrial autonomous vehicle tasks. Furthermore, the RL method tends to be unsafe and brittle to scenarios not encountered in training. Conversely, the optimization-based method offers a substantial level of explainability, and through the explicit inclusion of safety constraints, it can guarantee the system’s safety. In this context, building upon RL framework, a fusion algorithm that combines the advantage of the RL-based scheme and optimization-based scheme is proposed. Specifically, unlike traditional RL-based solutions that directly execute from perception to control using only neural network maps, this work introduces a mechanism of uncertainty-aware interval prediction to compute the set of states that can be reached over the planning time horizon. On this basis, a robust control framework is presented, which guarantees system safety while considering the worst case performance scenarios. To validate the proposed algorithm, the task of an autonomous vehicle merging on to a highway from an on-ramp is simulated in the simulation of urban mobility (SUMO). The results show that the proposed motion planning and control method combines the advantages of RL and optimization-based methods and achieves balanced performance in smoothness, computational efficiency, explainability, and robustness.