Temporal Logic Guided Meta Q-Learning of Multiple Tasks

Reinforcement learning (RL) based approaches have enabled robots to perform various tasks. However, most existing RL algorithms focus on learning a particular task, without considering generalization to new tasks. To address this issue, by combining meta learning and reinforcement learning, we develop a meta Q-learning of multi-task (MQMT) framework where the robot effectively learns a meta model from a diverse set of training tasks and then generalizes the learned model to a new set of tasks that have never been encountered during training using only a small amount of additional data. Particularly, the multiple tasks are specified by co-safe linear temporal logic specification. As a semantics-preserving rewriting operation, LTL progression is exploited to decompose training tasks into learnable sub-goals, which not only enables simultaneous learning of multiple tasks, but also facilitates reward design by converting non-Markovian reward process to Markovian ones. Reward shaping is further incorporated into the reward design to relax the sparse reward issue. The simulation and experiment results demonstrate the effectiveness of the MQMT framework.