Skip to Main Content
Reinforcement learning is a popular method for learning in autonomous dynamical systems. One of the most popular reinforcement learning methods is Qlearning, where the evaluation function and action selection function is combined in one data structure. However, Q-learning suffers from poor scalability and slow convergence, problems typically addressed by clustering of states or by using a hierarchical action system. Hierarchical Q-learning, presented in this paper, provides a simple mechanism for dynamical creation of hierarchical action sequences, that solves the scalability problems of regular Q-learning, while retaining its simplicity. By creating dynamical action sequences and using them to generalize over the statespace, the model is able to increase the learning speed without prior assumptions about the structure of the state-space.