Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-Part I: I.I.D. rewards

At each instant of time we are required to sample a fixed number
m \geq 1
out of
N
i.i.d, processes whose distributions belong to a family suitably parameterized by a real number
\theta
. The objective is to maximize the long run total expected value of the samples. Following Lai and Robbins, the learning loss of a sampling scheme corresponding to a configuration of parameters
C = (\theta_{1},..., \theta_{N})
is quantified by the regret
R_{n}(C)
. This is the difference between the maximum expected reward at time
n
that could be achieved if
C
were known and the expected reward actually obtained by the sampling scheme. We provide a lower bound for the regret associated with any uniformly good scheme, and construct a scheme which attains the lower bound for every configuration
C
. The lower bound is given explicitly in terms of the Kullback-Liebler number between pairs of distributions. Part II of this paper considers the same problem when the reward processes are Markovian.