Classifier-Based Policy Representation

Motivated by recent proposals that view a reinforcement learning problem as a collection of classification problems, we investigate various aspects of policy representation using classifiers. In particular, we derive optimal policies for two standard reinforcement learning domains (inverted pendulum and mountain car) in both deterministic and stochastic versions and we examine their internal structure. We then proceed in an evaluation of the representational ability of a variety of classifiers for these policies, using both a multi-class and a binary formulation of the classification problem. Finally, we evaluate the actual performance of the policies learned by the classifiers in the original control problem as a function of the amount of training examples provided. Our results offer significant insight in making the reinforcement-learning-via-classification technology successfully applicable to hard learning problems.