End-to-end object detection and recognition in forward-looking sonar images with convolutional neural networks

Object detection and recognition are typically stages that form part of the perception module of Autonomous Underwater Vehicles, used with different sensors such as Sonar and Optical imaging, but their design is usually separate and they are only combined at test time. In this work we present a convolutional neural network that does both object detection (through detection proposals) and recognition in Forward-Looking Sonar images and is trained with bounding boxes and class labels only. Convolutional layers are shared and a 128-element feature vector is shared between both tasks. After training we obtain 93% correct detections and 75% accuracy, but accuracy can be improved by fine-tuning the classifier sub-network with the generated detection proposals. We evaluated fine-tuning with a SVM classifier trained on the shared feature vector, increasing accuracy to 85%. Our detection proposal method can also detect unlabeled and untrained objects, and has good generalization performance. Our unified method can be used in any kind of sonar image, does not make assumptions about an object's shadow, and learns features directly from data.