Breaking the Inertial Thinking: Non-Blocking Multipath Congestion Control Based on the Single-Subflow Reinforcement Learning Model

The Multipath TCP (MPTCP) protocol has received more attention due to the increasing number of terminals with multiple network interfaces. To meet the higher network performance demand of terminal services, many researches leverage reinforcement learning (RL) for MPTCP congestion control (CC) algorithms to improve the performance of MPTCP. However, we observe two limitations of existing RL-based mechanisms that make them impractical: 1) Fail to break the restriction of the input and output dimensions of RL, making the mechanisms unadaptable to the varying number of subflows. 2) Frequent model decisions block packet transmission, leading to under-utilization of bandwidth. This paper breaks the inertial thinking1 to overcome the above limitations and proposes Maggey, a non-blocking CC mechanism that applies the single-subflow model to multipath transmission. To this end, Maggey employs loosely coupled design principles and a unique reward function to ensure the fairness of the algorithm. Additionally, Maggey introduces iterative training to ensure the accuracy of training of the single-subflow model. Furthermore, a mode transition framework is artfully designed to avoid blocking, preserving the flexibility of RL-based CCs. These two features enhance the practicability of Maggey and the paper analyze the stability of Maggey. We implement Maggey in the Linux kernel and evaluate the performance of Maggey through extensive emulation and live experiments. The evaluation results show that Maggey boosts 26% throughput over DRL-CC at high bandwidth and improves 2%-60% throughput over traditional algorithms under different network conditions. Besides, Maggey maintains fairness in different scenarios.