Learning control of finite Markov chains with periodically varying transition probabilities

A learning scheme is presented for a Markovian decision problem with estimation of unknown transition probabilities which are dominated by a periodically varying parameter with period T. According to this scheme, at every T time instant the unknown parameter is estimated and then a policy sequence to be applied at the next T time instant is determined. It is shown that the estimate converges to the true value as time evolves and accordingly this scheme asymptotically attains control which is β-optimal in a broad sense