Reconfigurable Intelligent Surface Aided Mobile Fog Computing: A Space Aggregation-based Lyapunov Driven Reinforcement Learning Approach

The rapid proliferation of mobile devices within Internet of Things (IoT) has substantially heightened the demand for mobile edge computing (MEC). Fog computing (FC) is a more advanced form of edge computing that allows computing nodes to cooperate with each other. Reconfigurable intelligent surfaces (RIS) have emerged as a critical technology for optimizing wireless communication environments, attracting considerable attention. In this paper, we develop an online optimization problem for RIS-aided mobile FC deployed across wireless networks with computing nodes at the base stations (BS). We propose a Lyapunov-drift-plus-penalty-based, space aggregation-assisted proximal policy optimization (LSAPPO) algorithm to tackle the challenges in online optimization problem in RIS-aided mobile FC system. Our technique integrates a reinforcement learning (RL) algorithm employing the proximal policy optimization (PPO) agent, further enhanced by Lyapunov drift-plus-penalty optimization. The space aggregation technique effectively consolidates excessive decision variables and channel state information (CSI) into a manageable set of parameters to streamline the computing framework. Numerical simulation result shows that our proposed algorithm surpasses the benchmarks, underscoring the effectiveness in complicated wireless networks. Furthermore, we introduce the multi-agent LSAPPO algorithm to address the distributed demands of practical scenarios. The multi-agent LSAPPO algorithm enhances convergence speed and performs better in large-scale problems.