RL meets Multi-Link Operation in IEEE 802.11be: Multi-Headed Recurrent Soft-Actor Critic-based Traffic Allocation

IEEE 802.11be -Extremely High Throughput-, commercially known as Wireless-Fidelity (Wi-Fi) 7 is the newest IEEE 802.11 amendment that comes to address the increasingly throughput hungry services such as Ultra High Definition (4K/8K) Video and Virtual/Augmented Reality (VR/AR). To do so, IEEE 802.11be presents a set of novel features that will boost the Wi-Fi technology to its edge. Among them, Multi-Link Operation (MLO) devices are anticipated to become a reality, leaving Single-Link Operation (SLO) Wi-Fi in the past. To achieve superior throughput and very low latency, a careful design approach must be taken, on how the incoming traffic is distributed in MLO capable devices. In this paper, we present a Reinforcement Learning (RL) algorithm named Multi-Headed Recurrent Soft-Actor Critic (MH-RSAC) to distribute incoming traffic in 802.11be MLO capable networks. Moreover, we compare our results with two non-RL baselines previously proposed in the literature named: Single Link Less Congested Interface (SLCI) and Multi-Link Congestion-aware Load balancing at flow arrivals (MCAA). Simulation results reveal that the MH-RSAC algorithm is able to obtain gains in terms of Throughput Drop Ratio (TDR) up to 35.2% and 6% when compared with the SLCI and MCAA algorithms, respectively. Finally, we observed that our scheme is able to respond more efficiently to high throughput and dynamic traffic such as VR and Web Browsing (WB) when compared with the baselines. Results showed an improvement of the MH-RSAC scheme in terms of Flow Satisfaction (FS) of up to 25.6% and 6% over the the SCLI and MCAA algorithms.