Emergent Discovery of Reinforced Programs using Q-Learning and Planning: A Proof of Concept

While applying genetic programming to reinforcement learning tasks, little attempt is made to incorporate local state specific reward information. This reduces the sample efficiency of the approach when tasks are rich in rewards. In this work, we provide a proof of concept for using local rewards under grid world tasks to incrementally parameterize teams of programs. A planning step is then introduced to propagate Q-values through the team of programs. No interaction with the task is necessary to perform the planning step. Benchmarking is performed on
$n\times n;n\in\{5,10,20,50\}$
grid world navigation tasks to illustrate the utility of the approach.