On the optimality of the Gittins index rule in multi-armed bandits with multiple plays

Investigates the general multi-armed bandit problem with multiple servers. The authors determine a condition on the reward processes sufficient to guarantee the optimality of the strategy that operates at each instant of time the projects with the highest Gittins indices. The authors call this strategy the Gittins index rule for multi-armed bandits with multiple plays, or briefly the Gittins index rule. The authors show by examples that: (i) the aforementioned sufficient condition is not necessary for the optimality of the Gittins index rule; and (ii) when the sufficient condition is relaxed the Gittins index rule is not necessarily optimal. Finally, the authors present an application of the general results to the multiserver scheduling of parallel queues without arrivals.