Optimally Myopic Scheduling Policy for Downlink Channels With Imperfect State Observation

We consider a downlink channel scheduling problem in a wireless communication system where a base station serves a user by a number N of multistate Markov channels. In each time slot one channel is allocated to the user according to imperfect state observation information, and some reward is accrued depending on the state of the allocated channel. The objective is to design a scheduling policy that maximizes the expected accumulated discounted reward over an infinite time horizon. Mathematically, the decision-making problem can be formulated into a restless multiarmed bandit with exponential memory and time complexity. One of the feasible policies is the myopic (or greedy) one, which only maximizes the immediate reward with linear complexity. Specifically, we carry out a theoretical analysis on the performance of the myopic policy with imperfect observation, introduce the monotone likelihood ratio order to characterize the evolving structure of information states, and establish a set of closed-form conditions to guarantee the optimality of the myopic policy in downlink channel allocation.