Deep Reinforcement Learning-based Joint Frame Length and Rate Adaption for WLAN Network

Frame aggregation and physical rate adaptation are the most important enhancement for Wi-Fi network. However, both of them involve certain tradeoffs between achieving higher throughput and facing a higher error rate. The gain suffers from the imperfect and highly dynamic channel condition. In addition, there is a certain coupling relationship between the aggregation frame length and the physical rate. That means the selection of physical rate may affects the optimal frame length, and vice versa. Therefore, a joint frame length and rate adaption scheme is needed. Moreover, the large number of all available frame lengths and rates makes the joint adaption more challenging. In this paper, we propose a joint frame length and rate adaption (JFRA) scheme based on Double Deep Q-learning (DDQN) algorithm. The proposed scheme can automatically explore the environment and learn the optimal frame length and rate from experience. We apply prioritized training and incorporate reward value into the computation of experience priority. It can improve learning efficiency and accelerate the convergence of JFRA. We implement and evaluate JFRA in ns3-ai framework and the simulation results show that JFRA can outperform the Minstrel HT and Thompson Sampling algorithm by up to 21.3% and 68.9% in various cases.