Reconfiguring Satellite CDNs With Dynamic Uncertain User Requests Based on Multi-Agent DRL

The Satellite-Terrestrial Integrated Network (STIN) is a key paradigm to achieve global coverage and ubiquitous connection in the 6G era. Integrating the network slice technology based on Software Defined Networking (SDN) and Network Function Virtualization (NFV) into STIN is recognized as an effective solution to achieve a rapid flexible service provisioning. Specifically, the Content Delivery Network (CDN) service, which is storage resource intensive, is suitable to be deployed on STIN to provide a global range content service supply. Most existing research on the resource deployment of STIN focuses on static requests, ignoring the dynamic changes in requests caused by the high-speed movement of satellites. Especially, the reconfiguration of CDN slices and the variation of STIN are asynchronous in different time scales: this makes the reconfiguration of CDN slices to cope with faster changing user requests a challenging task. In this paper, we adopt a periodic reconfiguration strategy to configure CDN slices on the edge LEO satellite network of STIN in discrete time intervals. Within each time interval, we formulate the reconfiguration optimization problem to cope with the dynamically changing user requests, wherein the Stochastic Network Calculus (SNC) is used to measure the deployment performance within this time interval. Then, we describe the optimization problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), and propose a reconfiguration algorithm based on Multi-agent Deep Reinforcement Learning (MADRL) to determine the optimal adjustment strategy. Finally, intensive simulations are implemented to verify the performance of the algorithm. Compared with the baselines, the QoS of the proposed algorithm is increased by about 5.8%, while the operation cost and reconfiguration cost are decreased by about 8.5% and 35.3% respectively.