Intelligent and Distributed Routing for Leo Satellite Networks: A Lyapunov Optimization Aided Deep Reinforcement Learning Approach

In the routing process of low earth orbit satellite networks, frequent topology changes and complex space environment cause routing interrupted and sudden link failures. Traditional terrestrial routing not only fails to manage this problem but also leads to random network congestion. To address these issues and to achieve high adaptability and stability routing strategy, this paper proposes an intelligent distributed routing algorithm based on multi-agent deep reinforcement learning (MADRL) with Lyapunov optimization. Firstly, we build Lyapunov-based network optimization model and analyze the model stability. Then, we combine the Lyapunov optimization with the MADRL framework to stabilize the network. Each satellite agent selects next hop node according to the queue backlogs and the distance between the next hop node and the destination. Evaluation results show that our proposed Lyapunov optimization aided DRL (LOA-DRL) algorithm has better performance in terms of delivery ratio, average delivery time, throughput and average queue backlogs.