Multiple Reconfigurable Intelligent Surfaces Aided Vehicular Edge Computing Networks: A MAPPO-Based Approach

Reconfigurable intelligent surface (RIS) is envisioned as a new technology to improve the quality-of-service in vehicular edge computing (VEC) networks due to its ability to change the wireless radio propagation environment. In this paper, we study multi-RIS-assisted VEC networks, where vehicle user equipments (VUEs) can offload tasks to nearby base stations (BSs) with integrated edge services (ESs) and select an efficient computation ES. Meanwhile, the individual virtual machine (VM), which is defined as a software clone of the VUE's service environment containing the profile and application to run the VUE's tasks, need to be migrated to the same ES to complete the process of the offloaded tasks. Accordingly, we formulate a throughput maximization for multi-RIS-assisted VEC networks via jointly optimizing the selected ESs for the VUEs, the deployment locations of RISs, and reflection matrices of RISs, subject to the maximum tolerable delay. To solve the non-convex mixed-integer nonlinear optimization problem, we propose an efficient algorithm based on the multi-agent proximal policy optimization (MAPPO) with the centralized training and decentralized execution (CTDE) framework, where both the BSs with integrated ESs and the VUEs are regarded as two types of heterogeneous agents interacting with the environment. In particular, several tricks such as reward normalization, orthogonal initialization, and learning rate decay are adopted to improve the convergence performance of the MAPPO-based algorithm. Numerical simulation results demonstrate the fast convergence behavior of the proposed MAPPO-based algorithm compared to other conventional multi-agent reinforcement learning (MARL) algorithms. Moreover, the results also show that the throughput obtained by the proposed MAPPO-based scheme outperforms that obtained by the scheme without multi-RIS 26.41% and that obtained by the scheme without service migration 23.65%, respectively. Besides, it is verified that deploying locations of RISs is a valid solution to enhance the throughput of VEC networks.