Lane Changing Using Multi-Agent DQN

This study explores the feasibility of autonomous lane changing using a novel approach of multi-agent Deep Q-Network. Most existing algorithms that use Deep Reinforcement Learning adopt a single-agent approach, with the assumption of only ego-agent changing lanes. We argue that such an approach is merely a simplification of the real-world without considering multi-agent negotiations. In this work, we model the lane changing problem as a multi-agent system and develop a decision-making policy using Deep Q-Network. We address the non-stationarity problem caused by our multi-agent setup which includes an Experience Replay. While prior research recommends avoiding the Experience Replay under such conditions, we report for the first time that an Experience Replay can help yield a robust negotiation policy in our lane changing experiment, without impairing the training of the Deep Q-Network. We show that our approach enables the model to learn negotiating-behaviors like overtaking, yielding, lane interchanging, and lane merging.