Heterogeneous Multi-Agent Reinforcement Learning for Channel Access in WLANs

This paper investigates the challenge of heterogeneous multi-agent reinforcement learning (MARL) algorithms in wireless local area networks (WLANs), where multiple stations utilize either value-based or policy-based reinforcement learning algorithms for channel access. Specifically, we propose a novel heterogeneous MARL training framework, named QPMIX, which adopts a centralized training with decentralized execution paradigm to enable heterogeneous agents to collaborate. Our method aims to maximize the network throughput and ensure fairness among stations, enhancing the overall performance of WLANs. Through the simulation results, we demonstrate that the proposed QPMIX algorithm achieves higher throughput, lower mean delay, reduced delay jitter, and decreased collision rates than conventional CSMA/CA in the saturated traffic scenario. Additionally, it can better promote cooperation between heterogeneous agents compared to independent learning.