Hypergraph-Based Resource-Efficient Collaborative Reinforcement Learning for B5G Massive IoT

Beyond 5G (B5G) networks rapidly growing to connect billions of Internet of Things (IoT) devices and the dense deployment of IoT devices leads the large-scale network conflict and obstacles the resource-efficient, which brings a great challenge for network resource management (NRM). To tackle this problem, hypergraph based resource-efficient collaborative reinforcement learning (CRL) was proposed for B5G massive IoT. Firstly, the hypergraph theory based network conflict model was formulated to quantify the conflict degree of the B5G massive IoT. Then, since the conflict-free resource management problem is a combinatorial optimization problem with NP-hard, the resource management based Markov decision process (MDP) model was built for NRM in B5G massive IoT. To reduce the computational load by distributing the training overhead throughout the entire B5G massive IoT and achieve distributed collaborative learning, the federated averaging advantage Actor-Critic (FedAvg-A2C) based resource management is proposed to handle the network conflict-free resource management problem and accelerate the training process. Simulation results show the proposed scheme has high network throughput and the resource-efficient in B5G massive IoT.