A Heterogeneous-Agent Deep Reinforcement Learning Approach for Dynamic Spectrum Access in Cognitive Wireless Networks

Spectrum resources are indispensable for the deployment of wireless communications and they are currently facing significant scarcity issues. Dynamic spectrum access (DSA) plays an important role in addressing this challenge. Machine learning approaches can be used in the development of spectrum access technologies in cognitive radios. However, heterogeneity of secondary users (SUs) has been overlooked in most previous studies, particularly in cognitive radio networks, where SUs possess varying observation spaces, transmit powers, and aggregation capabilities for idle channels. In this work, we establish a dynamic spectrum access system model with inhomogeneous SUs and propose a novel solution for multi-user dynamic spectrum access that integrates a heterogeneous agent deep reinforcement learning algorithm to address inhomogeneous challenges. Simulation results demonstrate that the heterogeneous-agent reinforcement learning (HARL) based dynamic spectrum access scheme outperforms other prevalent multi-agent deep reinforcement learning approaches. The proposed scheme effectively enhances network throughput, reduces collisions, and addresses the user-inhomogeneity issues.