On Effectiveness of Exploration Strategies in Deep Reinforcement Learning for Power Allocation in Multi-Carrier Wireless Systems

This paper presents a comprehensive study on the efficiency and effectiveness of exploration policies for deep reinforcement (DRL) algorithms with applications to the power allocation problem in multi-carrier wireless systems. We propose three distinct exploration functions, i.e., linear, fast and slow, to balance exploration and exploitation in the dynamic wireless environment. We analyze the effect of exploration on the initial training length as well as learning models' sum-rate performance and power violation probabilities. Our results indicate that the DRL algorithms with the proposed exploration functions reach close-to-optimal sum-rate performance within only 1000 training episodes (i.e., equivalent to 8.01 s) while satisfying the predefined power constraint of the base station.