Zero-Shot Detection Based on Dual-Task Balancing

Zero-shot object detection (ZSD), which aims to detect unseen objects, has garnered significant attention due to its potential for expanding the range of detectable classes via textual extensions. However, current methods often directly adapt techniques from zero-shot learning (ZSL), overlooking the unique challenges of ZSD, particularly the need to simultaneously address classification and localization tasks. In this article, we identify classification as the primary bottleneck in ZSD and propose two novel strategies to address it. First, from a visual instrumentation perspective, we introduce decoupled and dynamic detection heads to enhance classification performance, along with a novel adaptive branch learning network that selectively processes relevant information for classification. Second, to mitigate modality differences caused by data collection and measurement discrepancies, we integrate a cross-modal alignment module with a contrastive embedding (CE) loss, which improves the projection matrix of the classification branch and enables more robust adaptive learning. Extensive experiments on three standard datasets [COCO, ILSVRC, and visual genome (VG)] demonstrate that our approach significantly outperforms both vanilla and state-of-the-art zero-shot detectors in test accuracy.