MERCA: A Multi-Objective Optimization Traffic Light Control Model

In recent years, deep reinforcement learning (DRL) has demonstrated significant capabilities in adjusting traffic signal timing. However, designing a multi-objective reward structure and addressing the issue of reward sparsity remain major challenges for DRL. In this paper, the authors propose a new reinforcement learning model architecture called MERCA (Multi-objective Deep Deterministic Policy Gradient) for multi-objective reward optimization. MERCA constructs corresponding critic for each optimization objective, and updates the critics based on the aggregated evaluation values of all objectives to achieve multi-objective optimization. To address the reward sparsity issue, this paper further investigates reward shaping methods. MERCA learns an intrinsic reward function in a completely self-supervised manner, in addition to the external reward, and maximizes the agent's performance through exploration-based rewards. The paper compares MERCA with other baseline models using the traffic simulator SUMO. The experimental results demonstrate that MERCA exhibits better performance in conventional traffic metrics and has faster learning speed for the intelligent agent.