Learning Parallel Markov Chains over Unreliable Wireless Channels

This paper studies the problem of communications between aircraft and a control tower for aviation risk monitoring over wireless channels. The control tower needs to monitor the state of each aircraft in real time by receiving reports from the aircraft. Due to limited bandwidth, only a subset of aircraft can communicate with the control tower at the same time. This paper focuses on the problem of optimal scheduling of data transmissions to minimize the risk. We formulate the problem as learning states of parallel Markov chains where each Markov chain represents an aircraft, and the objective is to minimize the information entropy of all the aircraft. We propose an algorithm based on Whittle’s index and study the indexability of the problem for both single-state wireless channels and multi-state wireless channels. Our numerical evaluations show that our algorithm improves the accuracy of the estimations compared with the heuristic scheduling methods such as greedy and Round& Robin.