GFD-Retina: Gated Fusion Double RetinaNet for Multimodal 2D Road Object Detection

In the field of Advanced Driver-Assistance Systems, road traffic actors detection is a vital task in order to avoid human errors in driving. Unlike camera only-based convolutional neural networks for 2D object detection, multimodality using improve object detectors accuracy and robustness. In this paper, we propose Stacked Fusion Double RetinaNet (SFD-Retina) and Gated Fusion Double RetinaNet (GFD-Retina), two convolutional neural networks taking multimodal data (RGB, Depth from Stereo, Optical Flow, LIDAR Point Cloud) as input. These networks combine efficiently sensor specific properties by using both early fusion and middle fusion for detecting road objects and their 2D localization. Evaluation of SFD-Retina and GFD-Retina on the challenging KITTI object detection benchmark shows that using sensor fusion improve significantly object detection accuracy. Moreover, GFD-Retina with Gated Fusion Unit outperforms SFD-Retina with Stacked Fusion Unit, and obtain satisfying results against state-of-the-art algorithms.