Multi-cluster Cooperative Offloading for VR Task: A MARL Approach with Graph Embedding

Virtual reality (VR) technology has recently achieved notable success and been widely expected to interplay with more mobile multimedia services. To further enhance real-time immersive experience for VR applications, exploiting cooperative offloading among capable terminal devices should be emerged as an effective means. However, faced with diverse and surging mobile VR user requests, terminal-assisted offloading needs to support comprehensive cached content, ultra-low latency delivery, and continuous energy provisioning, to guarantee stringent quality of service requirements, which poses a critical challenge for resource-constrained terminals. Hence, this paper proposes a Cooperative Offloading framework for Terminal Clusters (named CO-TC), in which VR terminal clusters form several cooperation groups for sharing cached field of view (FoV) tiles and available computing resources to cooperatively perform FoV rendering and content delivery. To maximize energy efficiency in CO-TC, an optimization problem is formulated to jointly decide the task offloading and computing resource utilization. An intelligent offloading scheme is designed based on multi-agent reinforcement learning (MARL) specially using agent relation feature graph embeddings. Moreover, we theoretically prove the permutation invariance and convergence of the proposed algorithm and derive the optimal observation range of the agent to balance the performance gain and interaction overhead in the distributed MARL frame. Finally, simulation results show that the proposed offloading scheme outperforms other baselines in terms of VR service performance, including latency, energy consumption, and energy efficiency.