Simulation-Based Benchmarking of RL Algorithms for Adaptive Thermal Control in IoT-Enabled Smart Umbrella Systems

This paper presents a simulation-based benchmarking analysis of three reinforcement learning (RL) algorithms—Soft Actor-Critic (SAC), Deep Q-Network (DQN), and Proximal Policy Optimization (PPO)—for real-time thermal regulation in IoT-enabled smart umbrella systems. Motivated by the environmental challenges faced during the Hajj pilgrimage, where pilgrims are exposed to extreme heat and humidity, a custom simulation environment was developed to emulate realistic Hajj conditions. Each RL agent was trained and evaluated across multiple training horizons. Experimental results show that SAC consistently outperforms DQN and PPO in achieving stable thermal comfort while minimizing energy consumption. Compared to traditional on-policy and value-based methods, SAC demonstrated faster convergence, better adaptability to dynamic environmental shifts, and suitability for deployment on resource-constrained microcontroller platforms. These findings highlight the potential of entropy-regularized reinforcement learning (RL) in climate-adaptive, embedded AI systems for human-centric applications.