Real-time Bird’s-Eye-View Panoptic Segmentation for Monocular-based Indoor Navigation

Bird’s-Eye-View (BEV) segmentation is a essential technology for safe and efficient navigation. This is even more necessary in indoor driving, where there are dynamic and unstructured objects such as people and robot. However, since there is no way to generate training data, most of the researches have been conducted mainly in outdoor environments. In this paper, we propose an innovative approach to address this challenge. We automatically generate BEV training data for indoor environments based on the physics engine of a simulator. This eliminates the needs for tons of real data and virtual environments. We also propose a lightweight network architecture capable of BEV panoptic segmentation in real-time. Based on this network and simple image processing, our framework shows fast but also robust performance in real-world environments with no gap between simulation and reality. The network can inference at a speed of about 61 FPS on AGX Orin in FP16 mode. Furthermore, it outperforms existing algorithms in the semantic segmentation task, achieving 14% higher mIoU.