Exploiting 3D Spatial Relationships for Target-driven Visual Navigation

We propose a visual navigation system that uses a currently popular deep reinforcement learning algorithm to make reasonable decisions in complex virtual 3D environments. The goal of our model is to train an intelligent agent to move towards a target location given an egocentric view of the target and its current observation. Our end-to-end trained agent uses RGB and 3D spatial relationships to make it possible to handle large environments across multiple scenes and different targets. When compared to several baselines, we demonstrate that our method: (1) has higher accuracy to arrive at target locations, (2) generates better solutions with shorter paths for navigation tasks, and (3) generalizes to unseen scenes and targets.