Deep Reinforcement Learning for QoE-Aware Offloading in Space-Terrestrial Integrated Networks

Low Earth Orbit (LEO) satellite-based edge computing offers an innovative paradigm for offloading the computing tasks of resource-limited User Equipments (UEs) in the environment lacking terrestrial networks. By offloading a portion of the computing tasks to satellite-mounted servers in space, it can significantly reduce the energy consumption of the UEs while reducing the delay. However, most existing works focus on the performance and energy consumption from the system point of view, neglecting the dynamic Quality of Experience (QoE) of the UEs: the UEs tend to place a higher value on saving energy when their remaining battery level is low, while seeking a lower delay when the battery power is sufficient. To this end, we build a QoE-aware task offloading model in space-terrestrial integrated networks where the UEs can adjust the preference on the delay and energy consumption based on their remaining battery level. We then propose a Self-Adaptive Multi-Agent Deep Deterministic Policy Gradient (SA-MADDPG) task offloading scheme based on Deep Reinforcement Learning (DRL). SA-MADDPG learns the status of the satellite-mounted edge servers and makes different offloading decisions at different battery levels. When the battery level is high, UEs will be more likely to perform computation locally to reduce the delay; on the contrary, when the UE battery level is low, they will seek to offload the computation tasks to the satellites to save energy. Through simulation experiments, we demonstrate that SA-MADDPG can dynamically adjust the UE offloading decision based on their current battery level and effectively improve the user QoE.