Panoptic Segmentation from Stitched Panoramic View for Automated Driving

Precise object detection is crucial in automated driving. In contrast to lidar and radar sensors, cameras provide high-resolutional measurements at comparatively low cost. A state-of-the-art method for object detection using camera images is panoptic segmentation, which combines semantic and object instance information. Current public datasets use multiple cameras to cover a larger area of the environment. But, the limited field of view occludes objects. As a results, on the one hand, the correct dimensions of objects cannot be captured and, on the other hand, false detections can occur. Objects can also be detected multiple times in the overlapping image area. To track dynamic objects, duplicate detections must be filtered. Rather than directly segmenting all camera images individually, we first stitch them into a horizontal panorama. Using a stitched surround view avoids detection difficulties at the boundaries of the individual images. For this purpose, we leverage the EfficientPS pre-trained network architecture and adapt it for use with panoramic images. In our evaluation, we demonstrate the improvement in panoptic quality of a stitched surround view. In addition, we separately compare the panoptic quality in the overlapping image areas between the panorama and the individual images. Finally, we show further advantages of panoramic images in terms of inference time in runtime analysis.