K-Cluster Algorithm for Automatic Discovery of Subgoals in Reinforcement Learning

Options have proven to be useful to accelerate agent's learning in many reinforcement learning tasks, determining useful subgoals is a key step for agent to create options. A K-cluster algorithm for automatic discovery of subgoals is presented in this paper. This algorithm can extract subgoals from the trajectories collected online in clustering way. The experiments show that the K-cluster algorithm can find subgoals more efficiently than the diverse density algorithm and that the reinforcement learning with this algorithm outperforms the one with the diverse density algorithm and flat Q-learning