Learning Contention Window Selection in Age of Information-Oriented IEEE 802.11 Networks

We study the problem of selecting the contention window (CW) for age of information (AoI)-oriented IEEE 802.11 networks using deep reinforcement learning (DRL) techniques. AoI quantifies information freshness and is defined as the time elapsed since the generation of the latest received information update. As the number of stations increases, conventional CW selection methods defined in IEEE 802.11 standards, such as binary exponential backoff (BEB), fail to achieve a low network-wide average AoI. Therefore, we leverage DRL and propose a CW control algorithm based on deep deterministic policy gradient (DDPG) that can dynamically learn appropriate CW values under different network conditions. Simulation results demonstrate that our proposed DRL-based approach reduces the average AoI of the system by about 57.6% in a congested network with 50 stations compared to the conventional BEB algorithm. Furthermore, compared to the algorithm using throughput as the DRL reward function, our AoI-reward-function approach achieves a 20.4% improvement in information freshness, indicating the key difference between AoI and throughput.