Unveiling the Effects of Experience Replay on Deep Reinforcement Learning-based Power Allocation in Wireless Networks

Deep reinforcement learning has emerged as a pow-erful tool for dynamic power allocation, as it allows continuous learning and realtime responsiveness to environmental changes through its core element, experience replay. Experience replay involves two critical hyperparameters: the replay buffer and mini-batch sizes. While state-of-the-art solutions primarily concentrate on designing input features and reward-shaping methods, the impact of experience replay parameters on system performance has often been overlooked. This paper aims to address this gap by exploring the effects of experience replay parameters in the context of dynamic power allocation in multi-carrier wireless systems. To address the power allocation problem, we propose a multi-agent cooperative deep reinforcement learning framework. The results show that a minimum of 2000 experiences in the replay buffer is necessary for the proposed solution to outperform conventional approaches. Moreover, many obsolete experiences within a larger replay buffer slightly decrease system performance. Interestingly, the increase in batch size does not significantly affect the learning models' training time due to parallel execution, yet, it improves performance.