CLIP-DS: Incorporating DataStd and Similarity Networks into CLIP for Scene Recognition Task

Existing deep learning methods encounter challenges when confronted with the scene recognition task, particularly in cases with significant intra-category differences, primarily due to limited available data. To better address this problem, in this paper, we propose Contrastive Language-Image Pre-Training with DataStd and Similarity networks (CLIP-DS), a novel scene recognition method built upon CLIP architecture, which utilizes extensive pre-training knowledge. In addition, to comprehensively analyze the similarity sequence provided by CLIP, we enhance it with the specialized Similarity networks, comprising of a Multi-stream Multilayer Perceptron block. To improve the data space of similarity sequences and ensure stable training, we further propose the DataStd network with learnable parameters. To the best of our knowledge, this work proposes the first dataset for the challenging intra-category difference industrial scene recognition task, encompassing 120k images. The experimental results demonstrate that the proposed CLIP-DS outperforms the other methods, achieving an accuracy of 87.59% with fine-tuning on only 90 images. Ablation Studies also demonstrate the effectiveness of the proposed Similarity and DataStd networks and identify optimized-prompt combination as the most suitable prompt strategy.