Online Learning Algorithms for Mobile Data Offloading

Mobile data offloading has become a promising approach to meet the exponential traffic growth in the cellular Network. In this work, we propose a queue-theoretic-based approach to address the problem of offloading data across a mobile network operator (MNO) and small cell service provider (SSP). Specifically, we develop a MDP (Markov Decision Process) based framework to identify offloading algorithms to minimize the sum of queue lengths of MNO and SSP. We propose an online algorithm based on conducting value-iteration by incessantly estimating the arrival and service rates. We conduct a detailed simulation study to demonstrate the efficacy of the proposed algorithm in comparison with (i) the traditional Q-Learning algorithm and (ii) a heuristic that chooses to offload randomly (independently of the current state of the system).