Mode Selection and Resource Allocation in Device-to-Device Communications With User Arrivals and Departures

The pervasive increasing mobile devices and explosively increasing data traffic pose imminent challenges on wireless network design. Device-to-device (D2D) communication is envisioned to play a key role in the fifth generation cellular networks to efficiently support much larger and more diverse set of devices. This paper investigates the mode selection and resource allocation for D2D communications with dynamic user arrivals and departures. We formulate the optimal resource control problem to minimize the average energy consumption of flow transmission into an infinite horizon average reward Markov decision process. In order to deal with the well-known curse of dimensionality problem and facilitate distributed implementation, we approximate the mode selection Q-factor by the sum of per-queue mode selection Q-factors. Moreover, we apply distributive stochastic online learning to estimate the per-queue Q-factors. Simulation results show that the proposed approach outperforms various existing baseline algorithms.