DDPG Based Dynamic Scheduling Optimization Mechanism in Real Switch

Nowadays, more and more traffic with different service requirements is transmitted simultaneously in the network, resulting in network congestion. In order to improve network Quality of Service (QoS), traffic will be classified and processed according to its requirements. The switch uses fair queuing and its extension schemes to schedule it to reduce the packet loss rate. However, statistical data such as packet arrival rate cannot be accurately obtained in the switches, and discrete reinforcement learning cannot model dynamically changing traffic, making existing dynamic bandwidth allocation solutions challenging to implement. Meanwhile, the same weight configuration will obviously cause the packet loss rate to fluctuate, so it is challenging to design an effective reward function. This paper proposes a dynamic scheduling optimization mechanism based on Deep Deterministic Policy Gradient (DDPG), called DDSO, in an actual switch to schedule flows with different service requirements. Firstly, the flow scheduling problem is modeled as a Markov Decision Process (MDP), and DDPG is used to find the optimal queue weight configuration. Queue weight adjustment is a continuous control problem, and DDPG can solve this problem very well. Secondly, we introduce two one-dimensional arrays to design a new reward function, to reduce the influence of packet loss rate fluctuation on reward determination. At last, this mechanism is evaluated through a hardware testbed in two scenarios. The results show that DDSO can effectively schedule the flows with different service requirements and alleviate network congestion.