Object Detection in Dense and Mixed Traffic for Autonomous Vehicles With Modified Yolo

Autonomous vehicles rely on the accurate detection and recognition of objects in their surroundings, a critical requirement for safe operation, especially in congested traffic with diverse vehicle types. This study presents a novel dataset collected in various road conditions in Indonesia, and it focuses on the detection and classification of visual objects around autonomous vehicles. Object recognition is achieved through the use of YOLOv7-based deep learning, adapted to identify small, faint, and partially concealed objects. Key enhancements include the integration of a deformable layer and the transition from Non-Maximum Suppression (NMS) to Soft Non-Maximum Suppression (softNMS). The dataset comprises eight predefined custom classes commonly encountered in Indonesian traffic. We collected video data recordings of heavy traffic scenarios featuring a wide range of vehicle types as training and testing data. The object detection model is fine-tuned through transfer learning, with multiple learning configurations explored for comparison. Experimental results demonstrate that deep learning models trained with transfer learning outperform those trained from scratch. Specifically, the modified YOLOv7, referred to as YOLOv7-MOD, incorporates a deformable convolution layer for up-sampling, leading to a remarkable performance of 94.68% in Recall, 96.87% in Precision, and 95.76% in F1-score. The modification resulted in an additional performance increase of 1.05% on average compared to the original model. The findings indicate that YOLOv7-MOD enhances the precision of object detection and recognition compared to the original YOLOv7, making it a promising solution for autonomous vehicle perception systems.