Longitudinal Position Control for Highway On-Ramp Merging: A Multi-Agent Approach to Automated Driving

Highly automated driving requires effective handling of many complex scenarios. Here we study a specific important task of highly automated driving: merging into traffic from a highway ramp. We take a multi-agent view of this problem. We propose a simplified mathematical formulation that captures the most fundamental interactions involved in this complex scenario and show that collisions may not be universally avoidable. We then propose a multi-agent simulator based on a slightly more sophisticated version of the problem to study the interaction of a single vehicle pair: an on-ramp merger and an in-lane traffic vehicle. While simple, the simulator captures the fundamental interaction of the general complex on-ramp merge scenario. We apply single- and multi-agent Q-learning within the simulator as a way to try to infer and analyze the individually optimal behavior that each vehicle may reach for the merging task under different informational conditions governing vehicles' interaction. Our results suggest that a multi-agent approach can produce controllers for highway-ramp merging with lower collision rates than those produced via a single-agent approach, but only if the individual behavior of the in-lane traffic vehicle remains optimal. We also discuss implications of our results for future work on this problem.