Detecting the Unexpected: A Safety Focused Multi-Task Approach Towards Unknown Object-Segmentation and Depth Estimation

This work addresses the problem of object segmentation and pixel-wise distance (disparity) estimation in aMulti-Task Learning (MTL) framework, where semantic segmentation enjoys robustness against unexpected objects viaOut-of-Distribution (OoD) segmentation. The proposed MTL network, referred to as ’OSDNet’, leverages the positive transfer between in-distribution, OoD and disparity learning, while kept sufficiently less complex for real-time applications. This positive transfer resulted in 0.52% improvement in mean Intersection over Union (mIoU), 3% increase in Area Under the Precision Recall Curve (AUPRC) with 7% improvement in error (FPR95) of MTL in semantic segmentation and OoD detection, respectively, compared to Single-Task Learning (STL) approach for semantic segmentation task (’OSNet’ is used in STL approach which has only semantic segmentation head while rest of the architecture is same as in ’OSDNet’). Due to the missing semantic labels of publicly-available dataset for training, we propose a semi-automatic relabeling technique. In this work, we orient ourselves in both data-centric and model-centric approaches, i.e., a new set of data is derived for the learning process, and a novel model is proposed for the target MTL. The code and the resultant dataset are made publicly available at https://github.com/ravivaghasiya1998/MTLOSDNet.