Deep Reinforcement Learning Based Contention Window Optimization for IEEE 802.11 bn

The up-to-date project authorization request (PAR) for IEEE 802.11 bn envisions achieved Ultra High Reliability capability on the basis of the Extremely High Throughput Wi-Fi (IEEE 802.11 be). Specifically, it calls for optimizing the 95th percentile of the latency distribution and MAC Protocol Data Unit (MPDU) loss while ensuring high throughput. The vision is challenging due to the competing nature of Wi-Fi channel access, especially in the case of overlapping basic service set (OBSS). This challenge gives rise to an emerging research topic of Wi-Fi, i.e., low-latency channel access. In this paper, we materialize low-latency channel access via multi-agent reinforcement learning (MARL). To meet the Wi-Fi legacy requirement, we do not drop the carrier sense multiple access with collision avoidance (CSMA/CA) protocol but resolve to tune contention window (CW) being intelligent, a critical control parameter of the protocol. The objective of intelligent adapting CW is to minimize tail latency constrained by throughput and MPDU loss, which is consistent with the PAR. The control optimization problem is then solved by MARL. The adopted method promises effectiveness in distributed learning, which is validated in many simulations covering different topologies. Extensive simulation results show that this method has an average substantive gain of more than 25%.