Open World Object Detection Via Cooperative Foundation Models for Driving Scenes

Open-world object detection plays a vital role in ensuring the safety and reliability of autonomous vehicles, as it strives to identify unknown objects not appeared during inference and incrementally learn to detect new classes once they become labeled. However, existing works are trained and evaluated on generic benchmarks which exhibit a significant domain gap compared to driving scenes. In this paper, we introduce a novel Open-world object Detection algorithm for Driving scenes, named OpenDet-D, which incorporates dual branches to leverage remarkable generalization capability of foundation models to address the challenge of open-world driving scenes. We also introduce a novel metric designed for driving scenes to evaluate incremental learning. Extensive experiments on CODA2022 and SODA10M benchmarks demonstrate the effectiveness of our proposed method.