Enhancing Autonomous Driving with Grounded-Segment Anything Model: Limitations and Mitigations

Recent advancements have been observed in the domains of object detection and semantic segmentation, notably with the introduction of the Grounding DINO and Segment Anything Model (SAM). In the realm of autonomous driving, an effective model is crucial to accurately perceive the road environment, thus warranting the exploration of these innovative models. The Grounded-SAM, known as Grounded Segment Anything Model, is an advanced machine learning model that has been designed for object detection and semantic segmentation. This feature makes it an invaluable tool in domains like autonomous driving, where accurate real-time object detection and understanding of the driving environment are critical for safe and efficient operation. In this paper, we first work on exploring limitations of Grounded-SAM in autonomous driving, where we identified its weakness in differentiating objects in fine-grained tasks. To mitigate this limitation, we leverage zero-shot and pre-trained classifiers to perform traffic signs and vehicles detection to enhance the precision of Grounded-SAM in autonomous driving.