MAC Protocol for Multi-channel Heterogeneous Networks Based on Deep Reinforcement Learning

This paper considers the problem of efficient spectrum utilization in heterogeneous wireless networks (HetNets), wherein different radio networks adopt different medium access control (MAC) protocols to transmit data packets to a common access point on different wireless channels. To allow emerging radio nodes to transmit on those pre-allocated channels and to expedite more efficient spectrum utilization, we exploit the advanced deep reinforcement learning technique to develop a new generation of MAC protocols, referred to as multi-channel deep-reinforcement learning multiple access (MC-DLMA). The emerging radio nodes that adopt MC-DLMA can make full use of the underutilized spectrum resource and maximize the sum throughput of the overall HetNet by learning the transmission patterns of the existing radio nodes. For benchmarking, we derive the optimal throughputs analytically and demonstrate that MC-DLMA can achieve the near-optimal results. Moreover, compared with other baselines (e.g., the Whittle Index policy and the random access policy), our MC-DLMA can significantly improve the sum throughput of the HetNet in various scenarios.