Dual-Attention Model for Camera-Based Few-Shot Object Detection

Few-shot object detection (FSOD) is a developing research topic in computer vision. Its core idea is to leverage abundant samples from base categories to train a model, which can then be transferred to detect target categories using only a limited number of labeled samples. FSOD methods often face challenges such as inadequate feature extraction capabilities and limited generalization performance due to the scarcity of annotated samples. To effectively alleviate this issue, we integrated additional sources of information, including rich textual descriptions and intuitive image-aided information, to develop a novel model—the dual-attention model. Our model incorporates two attention-based modules: the dual encoder module (DEM) and the dual correlation module (DCM). The DEM enhances spatial features from the backbone using contrastive language-image pretraining (CLIP) image features. Its segmentation and detection branches, which are based on an encoder, form a mutually reinforcing relationship through cross-branch fusion. The DCM employs a cross-modal fusion strategy to integrate CLIP text features with both the global features produced by the backbone and the output features of DEM. It utilizes an attention mechanism to further enhance the text-enhanced DEM output features by incorporating text-enriched global features. To assess the effectiveness of our model, we conducted thorough evaluations on the PASCAL VOC and MS COCO datasets. The experimental results showed that our model achieved competitive detection accuracy compared to other methods.