An Efficient Parameter-Free Learning Automaton Scheme

The learning automaton (LA) that simulates the interaction between an intelligent agent and a stochastic environment to learn the optimal action is an important tool in reinforcement learning. Being confronted with an unknown environment, most learning automata have more than one parameters to be tuned during a pretraining process in which the LA interacts with the environment. Only after the parameters are tuned properly, an LA can act most properly during the training procedure to obtain the optimal behavior. The cost of parameter tuning can be enormous, e.g., possibly millions of interactions are required to seek the best parameter configuration. Therefore, the parameter-free LA that uses identical parameters for every environment and saves further tuning has become the hot spot of this research. This article proposes an efficient parameter-free learning automaton (EPFLA) that depends on a separating function (SF). Taking advantage of both frequentist inference and Bayesian inference, the SF plays a dual role in the proposed scheme: 1) evaluating the difference in performance between actions in the environment and 2) exploring actions by coining an action selection strategy. A proof is provided to ensure the
$\epsilon $
-optimality of EPFLA. Comprehensive comparisons verify the privileges of EPFLA over both parameter-based schemes and existing parameter-free schemes.