Joint Multiclass Object Detection and Semantic Segmentation for Autonomous Driving

Object detection and semantic segmentation are two fundamental problems in autonomous driving systems. As recent studies have illustrated the strong correlation between the two tasks, the joint development of object detection and semantic segmentation tasks by utilizing end-to-end encoder-decoder architecture has gained popularity in recent years. However, context information loss is a common problem for simple encoder-decoder systems. Considering this problem, this study proposes a joint multi-class object detection and semantic segmentation method with the addition of a modern feature fusion mechanism to prevent context information loss. The experiments are conducted on our proposed large-scale Inha Computer Vision 2022 (ICV22) dataset that was specifically collected and annotated for object detection and semantic segmentation tasks. The proposed model achieves 87.2 mIoU performance on the introduced ICV22 dataset with 92.1 accuracy for the segmentation task, which is far superior to that of the DeeplabV3++ semantic segmentation method. Additionally, joint object detection and semantic segmentation model illustrated 40.2 mAP for object detection task and 56.4 mIoU for semantic segmentation task, outperforming previously introduced methods on publicly available Cityscapes dataset with real-time inference speed of 42 FPS on NVIDIA RTX 3090 GPU.