On the MDP-Based Cost Minimization for Video-on-Demand Services in a Heterogeneous Wireless Network with Multihomed Terminals

In this paper, we deal with a cost minimization problem for a multihomed mobile terminal that downloads and plays a video-on-demand (VoD) stream. The cost consists of the user's dissatisfaction due to playback disruptions and communication cost for downloading the VoD stream. There are three components in our approach: parameter estimation, threshold adjustment, and threshold compensation. Since we do not assume any a priori knowledge about underlying random variables, necessary parameter values are estimated online. Using the resultant estimates, we formulate the problem as a Markov decision process (MDP) problem considering as if the random variables are exponentially distributed. To solve the MDP problem efficiently, we propose a threshold adjustment algorithm that exploits some structural properties of any optimal solution that are specific to our problem. Finally, to handle the cases where the random variables are not exponentially distributed, we propose a threshold compensation algorithm to compensate for the modeling error. Through extensive simulations, we compare the performance of our scheme with those of static threshold schemes.