Cooperative Multigroup Broadcast 360° Video Delivery Network: A Hierarchical Federated Deep Reinforcement Learning Approach

With the stringent requirement of receiving video from the unmanned aerial vehicle (UAV) from anywhere in the stadium of sports events and the significant-high per-cell throughput for video transmission to virtual reality (VR) users, a promising solution is a cell-free multi-group broadcast (CF-MB) network with cooperative reception and broadcast access-points (AP). To explore the benefit of broadcasting user-correlated decode-dependent video resources to spatially correlated VR users, the network should dynamically schedule the video and cluster APs into virtual cells for a different group of VR users with overlapped video requests. By decomposing the problem into scheduling and association sub-problems, we first introduce the conventional non-learning-based scheduling and association algorithms, and a centralized deep reinforcement learning (DRL) association approach based on the rainbow agent with a convolutional neural network (CNN) to generate decisions from observation. To reduce its complexity, we then decompose the association problem into multiple sub-problems, resulting in a networked-distributed Partially Observable Markov decision process (ND-POMDP). To solve it, we propose a multi-agent deep DRL algorithm. To jointly solve the coupled association and scheduling problems, we further develop a hierarchical federated DRL algorithm with scheduler as meta-controller, and association as the controller. Our simulation results show that our CF-MB network can effectively handle real-time video transmission from UAVs to VR users. Our proposed learning architecture is effective and scalable for a high-dimensional cooperative association problem with increasing APs and VR users. Also, our proposed algorithms outperform non-learning based methods with significant performance improvement.