Multi-agent Reinforcement Learning for Safe Driving in On-ramp Merging of Autonomous Vehicles

Although technology advances these years, the optimization problem of coordinating autonomous merging high-ways is challenging. This paper aims to achieve a smooth and safe traffic flow among autonomous vehicles with a proposed multiagent reinforcement learning (MARL) algorithm. Existing reinforcement learning algorithms converge to an arbitrary Nash equilibrium (NE) among multiple existing ones. However, multiple Stackelberg equilibrium exists in a small range of games. Besides, the Stackelberg equilibrium is better in Pareto performance than the average NE. In the general-sum Markov games, we take the autonomous vehicles on the main road as the leader agents while the others on the auxiliary road as the follower agents. We propose two curiosity-inspired MARL architectures to understand cooperation patterns and avoid being stuck in local-optimal collaborative solutions. Based on simulations, we prove that our proposed algorithms can perform better than benchmarks in reducing traffic jams.