Dynamic Controller Placement for Software-Defined LEO Network Using Deep Reinforcement Learning

In the context of satellite communication, software-defined satellite networking (SDSN) is deemed as a promising paradigm to remarkably boost the network flexibility and programmability. However, the implementation of the practical SDSN has been shadowed in ambiguity due to three problems by our analysis: 1) interdependent problem of controller placement (CP) and link mappings design, 2) devising a CP scheme which can handily deal with the partially-known traffic and attain the long-term performance boost, 3) Dynamically adjust the controllers’ position when the topology has changed. In order to solve these three tangled problems, we propose a dynamic CP algorithm to jointly decide the positions of multiple controllers and controller-switch mappings. Since in long term network optimization, the correlation between time slots is hard to mathematically formulate. Multi-agents deep Q-learning (MADQN) is adopted and jointly optimize switching cost, flow setup delay, and load balance from long-term perspectives. Finally, various experiments are conducted to compare MADQN with the prevalent K-means. MADQN is able to improve the performance of link switch number and load balance by 48.25%, 30.92% in average while nearly preserving low flow setup delay, which demonstrates the superiority of our scheme in boosting overall network performance.