Incentivized Exploration of Non-Stationary Stochastic Bandits

We study incentivized exploration for the multiarmed bandit (MAB) problem with non-stationary reward distributions, where players receive compensation for exploring arms other than the greedy choice and may provide biased feedback on the reward. We consider two different non-stationary environments: abruptly-changing and continuously-changing, and propose incentivized exploration algorithms accordingly. We show that the proposed algorithms achieve sublinear regret and compensation over time, and thus effectively incentivize exploration despite the nonstationarity and the biased or drifted feedback.