Real-time bus holding control on a transit corridor based on multi-agent reinforcement learning

Bus holding is the most used control strategy to improve bus service reliability. This paper presents a coordinated holding control framework based on multi-agent reinforcement-learning (MARL-H) to mitigate bus bunching in real-time on a bus corridor. The MARL-H framework depicts a combination of agents and methods to make decision of coordinated holdings. Coordination Graphs (CGs) are introduced in automatically selecting the coordinated holding actions in case of multiple buses dwelling at stops. Sparse Cooperative Q-learning algorithms are developed to obtain the joint coordinated holding actions due to the particularity of the sparsely structured graphs. The performance of the proposed method is evaluated by simulations. The results show the advantages of the MARL-H framework for the coordinated holding control in real-time. The proposed method could be embedded in an advanced public transportation system to improve bus operation performance.