Self-Supervised Occlusion Detection and Avoidance using Differentiable Rendering

Deep Learning-based computer vision methods have achieved state-of-the-art performance in the last decade, prompting a renaissance in the field of Artificial Intelligence (AI). Still, despite recent successes, many challenges remain including the robustness of these solutions to disturbances of the input, such as occlusion. While some solutions exist for neural network-based object detection under partial occlusions, these methods still implement the object detector as a passive observer in the scene. We argue, however, that an active object detector agent may be able to achieve better results by changing its own position to resolve occlusion, when possible. In this paper, an OpenAI Gym-compatible virtual environment is presented that enables the creation of realistic occlusion datasets via differentiable rendering. A trained neural network is also presented that predicts the occlusion mask and the optimal camera movement via self-supervised learning in the environment. Our experiments presented in this paper show that the neural network model is able to outperform the gradient-based optimization, and efficiently avoid occlusion in simulated scenes containing multiple objects.