Multiuser scheduling in a Markov-modeled downlink environment

We address the problem of multiuser scheduling in a cellular downlink system with partial channel information. In our setting, the channel of each user is modeled by a two-state Markov chain. The scheduler indirectly estimates the channel via accumulated automatic repeat request (ARQ) feedback from the scheduled users and uses this information in future scheduling decisions. This problem is a special case of the restless multi-armed bandit processes that have been shown to be PSPACE-hard to solve in general. By modeling the scheduling problem as a partially observable Markov decision process (POMDP), we formulate a throughput maximization problem and show that, despite the visible complexity of this problem, a simple round-robin fashioned scheduling policy optimizes the system for the special case of three or less users in the system. We study the structure of this policy for an arbitrary number of users and establish a sufficient condition for the optimality of this policy. Drawing equivalence with a genie-aided system, we derive an explicit expression for the sum capacity of the downlink.