Exploration Strategies in Adaptive Traffic Signal Control: A Comparative Analysis of Epsilon-Greedy, UCB, Softmax, and Thomson Sampling

Efficient traffic signal control is pivotal for managing urban transportation, ensuring a seamless traffic flow, and prioritizing safety. Decision-making strategies in dynamic environments, particularly the delicate balance between exploration and exploitation, are of paramount interest. This paper delves into exploration strategies and their profound impact on adaptive traffic signal control. The study thoroughly investigates four well-established strategies Epsilon-Greedy, Upper Confidence Bound (UCB), Softmax, and Thomson Sampling. These approaches offer unique perspectives on navigating the exploration-exploitation trade-off, specifically tailored for traffic signal control scenarios. Epsilon-Greedy makes probabilistic action selections based on a specified epsilon value, reserving the remaining probability for well-known actions. UCB employs uncertainty estimates to guide exploration, Softmax determines action probabilities using a softmax function based on expected rewards, and Thomson Sampling utilizes a Bayesian approach, sampling actions according to posterior distributions. Understanding the implications of these exploration strategies in the context of adaptive traffic signal control is crucial, given the intricacies of urban traffic environments and the evolving needs of modern cities. The paper’s findings contribute valuable insights to address challenges in urban transportation and congestion management, offering a nuanced perspective on optimizing traffic signal control through exploration strategies.