RotInvMTL: Rotation Invariant MultiNet on Fisheye Images for Autonomous Driving Applications

Precise understanding of the scene around the car is of the utmost importance to achieve autonomous driving. Convolutional neural networks (CNNs) have been widely used for road scene understanding in the last few years with great success. Surround view (SV) systems with fisheye cameras have been in production in various cars and trucks for close to a decade. However, there are very few CNNs that are employed directly on SV systems due to the fisheye nature of its cameras. Typically, correction of fisheye distortion is applied to the data before it is processed by the CNNs, thereby increasing the system complexity and also reducing the field of view (FOV). In this paper, we propose RotInvMTL: a multi-task network (MTL) to perform joint semantic segmentation, boundary prediction, and object detection directly on raw fisheye images. We propose a rotation invariant object detection decoder that adapts to fisheye distortion and show that it outperforms YOLOv2 by 9% mAP. By combining the MTL outputs, an accurate foot-point information and a rough instance level segmentation may be obtained, both of which are critical for automotive applications. In conclusion, RotInvMTL is an efficient network that performs well for autonomous driving applications.