Beam scheduling for electronically scanned array tracking systems using multi-arm bandits

We derive optimal and sub-optimal beam scheduling algorithms for electronically scanned array tracking systems. We formulate the scheduling problem as a multi-arm bandit problem involving hidden Markov models. A finite dimensional optimal solution to this multi-arm bandit problem is presented. The key to solving any multi-arm bandit problem is to compute the Gittins index. We present, a finite dimensional algorithm which computes the Gittins index. Suboptimal algorithms for computing the Gittins index are also presented.