Adaptive Traffic Management in Smart Cities using Deep Reinforcement Learning for IoT Mobile Ad Hoc Networks

This research evaluates four Deep Reinforcement Learning (DRL) algorithms-TD3, SAC, Rainbow DQN, and TRPO-for their effectiveness in adaptive traffic management, specifically in optimizing parking space management in smart cities. The algorithms were assessed based on key performance metrics: TD3 achieved a reward score of 85.6 with Actor Loss of 0.023, Critic Loss of 0.041, and 72 hours of training time. SAC outperformed with a reward score of 88.2, Actor Loss of 0.018, Critic Loss of 0.035, and 68 hours of training. Rainbow DQN achieved a competitive reward score of 87.4, Q-value of 0.075, Loss of 0.027, and 80 hours of training. TRPO attained a reward score of 86.8, Policy Loss of 0.032, Value Function Loss of 0.021, and 75 hours of training. SAC has shown best performance across the board, achieving a good exploration and exploitation trade-off. While Rainbow DQN can be seen excelling on tasks with discrete action spaces, the TD3 and TRPO clearly outperformed the others from continuous control task performance and a robust stable policy update of following iterations perspective respectively. The results show that SAC had the best overall performance with demonstrating a good trade-off between exploration and exploitation. While Rainbow DQN was robust to dilution in discrete action spaces, TD3 and TRPO performed well on continuous control tasks and stable policy updates respectively. These results give further evidence to the potential of DRL for real-world applications, such as dynamically assigning parking slots and directing traffic in cities towards efficient city-wide mobility. Future research may examine the further integration of real-time IoT data and increased scalability for citywide implementations to deliver smarter, more adaptive, traffic management solutions that account for the complexities of today's urban areas.