Rethinking the Reward Design for DRL in Mobile Communication Systems with Generative Diffusion Models

When applying deep reinforcement learning (DRL) in mobile communication systems, an appropriate reward function can effectively guide the agent towards learning the optimal action. However, it is challenging to accurately design a reward function in the dynamic and uncertain systems. To address the limitations when the reward function is unknown, suboptimal, or involves multiple objectives, we propose the Generative Diffusion Model-based Reward Exploration DRL (GRE-DRL) framework, which includes an exploration module in addition to the designed rewards. The proposed framework is evaluated in the context of a full-duplex (FD) mobile communication system based on deep deterministic policy gradient (DDPG) as the fundamental DRL approach, focusing on improving the performance of beam-forming and power allocation through the specific generative artificial intelligence (GenAI) models (i.e., generative diffusion models (GDM)). Numerical results demonstrate that the proposed algorithm outperforms conventional reward design methods by achieving faster convergence and reducing computation consumption, proving the effectiveness of our approach.