A Constrained MDP-Based Vertical Handoff Decision Algorithm for 4G Wireless Networks

The 4th generation (4G) wireless communication systems aim to provide users with the convenience of seamless roaming among heterogeneous wireless access networks. To achieve this goal, the support of vertical handoff in mobility management is crucial. This paper focuses on the vertical handoff decision algorithm, which determines under what criteria vertical handoff should be performed. The vertical handoff decision problem is formulated as a constrained Markov decision process (CMDP). The objective is to maximize the expected total reward of a connection subject to the expected total access cost constraint. In our model, a benefit function is used to assess the quality of the connection, and a penalty function is used to model signaling and call dropping. The user's velocity and location information are considered when making the handoff decisions. The value iteration and Q-learning algorithms are used to determine the optimal policy. Numerical results show that our proposed vertical handoff decision algorithm outperforms another scheme which does not consider the user's velocity.