Guided Deep Reinforcement Learning in the GeoFriends2 Environment

In recent years, the artificial intelligence community has taken big strides in the application of reinforcement learning to games or similar environments using deep learning. From Atari to board games, including motor control or riddle solving, fairly generic deep learning algorithms can now achieve great policies by simply learning to play from experience, and minimal knowledge of the specific domain. However, these algorithms are very demanding in terms of time and hardware in order to achieve the results reported in the literature. So much so, that some algorithms would take years to achieve state-of-the-art performance in commodity hardware. Not only that, but even the learning environments can hinder the speed of the learning process, if they have not been performance optimized. In this paper, we evaluate a complex existing environment, and propose a performance-oriented version, which we call GeoFriends2. We describe the motivation behind the creation of our version, and how it is suitable for both single- and multi-agent reinforcement learning. We then use Asynchronous Deep Learning to create complex policies that can act as baselines for future research on this environment. We also describe a set of techniques that speed up the learning process such that tests can be run with commodity hardware in hours, and not weeks, and using much simpler network architectures.