Enhancing Intersection Signal Control: Distributional Double Dueling Deep Q-learning Network with Priority Experience Replay and NoisyNet Approach

Aiming at the problems that most of the existing researches use a single indicator as a reward function for evaluating the behavior of intelligences as well as ignoring the efficiency of data sampling in the experience pool, which leads to unstable training process and slow convergence. In this paper, we propose a traffic signal control method based on the D4QN_PEN (Distributional Double Dueling Deep Q-learning with Priority Experience Replay and NoisyNet) deep reinforcement learning model. The method adopts a multi-indicator weighted average approach to define the reward function and makes the following improvements: (1) Use the Priority Experience Replay technique to give priority to the samples and increase the probability of occurrence of valuable samples. (2) Enhance the exploration ability of the model by introducing noise parameters through NoisyNet technique. (3) En-hance the model performance by utilizing Double Q Network, Dueling DQN and Distributional DQN. Additionally, the high-dimensional real-time traffic information of the intersection is converted into a state matrix consisting of vehicle positions, speeds and waiting times by variable-length discrete traffic coding, and the traffic state is abstractly characterized by deep learning to achieve adaptive signal control. To validate the proposed algorithm, it is verified on the simulation platform SUMO, and compared with the traditional DQN and Double Dueling DQN algorithms, the pro-posed D4QN_PEN algorithm is able to shorten the waiting time and queue length of the vehicles, and has a better control effect.