Stochastic online learning under unknown time-varying models

An online learning problem under stochastic time-varying models is considered. The problem is treated as a generalization of the classic multi-armed bandit problem when the arm distributions are time-varying. The objective is to study the impact of time variation in arm distributions on the performance of the player's strategy. Sufficient conditions on the rate of model variations under which learning can or cannot improve the regret order are established.