Sampling-Based Linear Approximate Planning for Underwater Space-Time Fair Scheduling

This paper investigates scheduling in space and time domains for multi-user underwater acoustic networks under fairness considerations. The problem is formulated as a sequential decision-making problem under the Markov Decision Processes (MDP) framework. Considering the difficulty of collecting data samples in an underwater acoustic channel for exploration, a planning approach is taken instead of online learning. To guarantee fairness among users, the proportional fair measure is employed, which breaks the additive structure between current and future rewards. To this end, a new, fairly weighted, decomposable reward function is proposed, enabling dynamic programming as the solution strategy. Furthermore, a sampling-based approximate planning scheme is developed to resolve the high computation complexity induced by the exponentially large state space. The characteristics of error accumulation in successive approximations are analyzed, and an upper bound on the approximation error is derived. It is shown that the instantaneous error decays with time. Numerical results show that the proposed scheme significantly improves network capacity while maintaining a high level of fairness relative to other schemes.