Graphic Deep Reinforcement Learning for Dynamic Resource Allocation in Space-Air-Ground Integrated Networks

Space-Air-Ground integrated network (SAGIN) is a crucial component of the 6G, enabling global and seamless communication coverage. This multi-layered communication system integrates space, air, and terrestrial segments, each with computational capability, and also serves as a ubiquitous computing platform. An efficient task offloading and resource allocation scheme is key in SAGIN to maximize resource utilization efficiency, meeting the stringent quality of service (QoS) requirements for different service types. In this paper, we introduce a dynamic SAGIN model featuring diverse antenna configurations, two timescale types, different channel models for each segment, and dual service types. We formulate a problem of sequential decision-making task offloading and resource allocation. Our proposed solution is an innovative online approach referred to as graphic deep reinforcement learning (GDRL). This approach utilizes a graph neural network (GNN)-based feature extraction network to identify the inherent dependencies within the graphical structure of the states. We design an action mapping network with an encoding scheme for end-to-end generation of task offloading and resource allocation decisions. Additionally, we incorporate meta-learning into GDRL to swiftly adapt to rapid changes in key parameters of the SAGIN environment, significantly reducing online deployment complexity. Simulation results validate that our proposed GDRL significantly outperforms state-of-the-art DRL approaches by achieving the highest reward and lowest overall latency.

I. Introduction

The Space-Air-Ground integrated network (SAGIN) is the key enabler of the 6G network to achieve ubiquitous global coverage and seamless connection [1]. This innovative framework spans space, air, and terrestrial domains, consisting of interconnected low earth orbit (LEO) satellites, aerial platforms (e.g., unmanned aerial vehicles (UAVs) or high altitude platform stations (HAPSs)), and ground-based networks [2]. In particular, the advancements in onboard processing and manufacturing cost reduction have significantly evolved LEO and aerial platforms from their traditional roles as mere transmission relays to fully-fledged computing platforms [3], [4]. This could provide a significant leap forward in a large number of emerging applications, such as hyperspectral sensing, intelligent transportation systems (ITS), and massive Internet of Things (IoT) [5], [6], [7].