MF-YOLO: Multimodal Fusion for Remote Sensing Object Detection Based on YOLOv5s

Remote sensing object detection has flourished at a rapid rate of development. In remote sensing images (RSI), most algorithms perform well in detecting medium and large objects but perform poorly on small objects. This is because small objects occupy fewer pixels in the entire image, and the detector is more likely to be biased towards detecting large objects. In addition, the huge background in the image also introduces noise, causing many detection failures. Most approaches solve the problem of small object detection by increasing the depth of the neural network, but the results are not satisfactory. To solve the above problems, we propose a new method, MF-YOLO, that utilizes fused infrared (IR) and red-green-blue (RGB) images for remote sensing object detection. Through multimodal fusion (MF), we can obtain more positive information, thereby improving detection accuracy. In addition, we introduced the Bi-Level Routing Attention (BRA) module to improve the YOLOv5s model structure, and proposed a new loss function to enhance the model’s learning discrimination ability. Experimental results show that the accuracy of MF-YOLO on the VEDAI and NWPU VHR-10 datasets is 76.62% and 91.63% respectively (in terms of mAP
@0.5
). Compared with the existing state-of-the-art, the detection accuracy is significantly improved. In addition, our model has fewer weight parameters than YOLOv5s, allowing it to achieve relatively high near-real-time detection speed.