Deep Reinforcement Learning Based Link Adaptation Technique for LTE/NR Systems

Outdated channel quality indicator (CQI) feedback causes severe performance degradation of traditional link adaptation (LA) techniques in long term evolution (LTE) and new radio (NR) systems. This paper puts forth a deep reinforcement learning (DRL) based link adaptation (LA) technique, referred to as deep reinforcement learning link adaptation (DRLLA), to select efficient modulation and coding scheme (MCS) in the presence of the outdated CQI feedback. The goal of DRLLA is to maximize the link throughput while achieving a low block error rate (BLER). We first give explicit definitions of state, action, and reward in DRL paradigms, thereby realizing DRLLA. Then, to trade off the throughput against the BLER, we further develop a new experience replay mechanism called classified experience replay (CER) as the underpinning technique in DRLLA. In CER, experiences are separated into two buckets, one for successful experiences and the other for failed experiences, and then a fixed proportion from each is sampled to replay. The essence of CER is to obtain different trade-offs via adjusting the proportion among different training experiences. Furthermore, to reduce the signaling overhead and the system reconfiguration cost caused by frequent MCS switching, we propose a new action selection strategy termed as switching-controlled
$\epsilon$
-greedy (SC-
$\epsilon$
-greedy) for DRLLA. Simulation results demonstrate that compared with the state-of-the-art OLLA, LTSLA, and DRLLA with other experience replay mechanisms, DRLLA with CER can achieve higher throughput and lower BLER in various time-varying scenarios, and be more robust to different CQI feedback delays and CQI reporting periods. Furthermore, with the SC-
$\epsilon$
-greedy policy, DRLLA can capture better trade-offs between the link transmission quality and the MCS switching overhead compared with other baselines.