Comprehensive Performance Analysis of Object Detection Algorithms Across Diverse Sensor Modalities

Object detection is a cornerstone of modern computer vision, driving advances in autonomous driving, robotics, surveillance, and smart infrastructure. However, detection performance and generalizability depend heavily on sensor modality and dataset characteristics. In this systematic review, conducted in accordance with preferred reporting items for systematic reviews and meta-analyses guidelines, we provide a comprehensive analysis of state-of-the-art object detection methods across diverse sensor types, including red, green, blue (RGB) cameras, light detection and ranging (LiDAR), radar, thermal, and depth sensors. We trace the evolution of benchmark datasets tailored to these modalities, examine how annotation strategies and sensor-specific features shape research directions, and highlight the growing adoption of multimodal and cross-modal datasets. Our findings reveal that, although object detection has traditionally relied on simple RGB images, recent years have seen a growing shift toward multimodal and cross-modal datasets, with 15% –20% of algorithms now incorporating LiDAR 3-D data and 10% –15% utilizing multimodal inputs. Furthermore, we analyze and compare the performance of state-of-the-art detection algorithms across six publicly available datasets. Our findings show that you only look once v3, for instance, achieves a recall of 98.7% and a precision of 84.77% on RGB frames from Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI), though its performance declines on more diverse datasets, such as JUVDsiv1. Voxel regional convolutional neural network yields 92% average precision on KITTI’s easy difficulty level, 82% on moderate, and 80.04% on hard levels for the car class using LiDAR 3-D data. Our study also reveals that multimodal methods consistently improve detection accuracy and generalization across challenging datasets, outperforming traditional single-modal approaches. In particular, multimodal sensor fusion enhances performance on complex benchmarks, such as KITTI and nuScenes. Finally, for 2-D image data, rotation sensitivity detection network (RSDet) provides strong results on the forward-looking infrared and low-light visible-infrared paired multimodal datasets. These findings highlight the advantages of multimodal and cross-modal approaches and underscore the need for future research into sensor-agnostic frameworks, real-time efficiency, scalable deployment on resource-constrained edge devices, and data-efficient learning strategies for emerging sensing modalities.