Ta-CLIP: A Framework for Chinese Cross-Modal Retrieval Using Text-Aware and Contrastive Learning

The key to improving semantic alignment and retrieval quality in image-text information lies in the precise understanding of key text information. Therefore, images should prioritize the most salient details emphasized in text descriptions, as users often rely on keywords for search. However, existing models face challenges in capturing the subtle corre-lations between vision and text semantics, often overlooking the importance of text perception. To address these issues, we introduce an efficient framework called Ta-CLIP, which enhances image perception of key text semantics by focusing on entities articulated in the text. Additionally, a dual-tower Transformer architecture is employed to synergize vision and text knowledge, and contrastive learning is incorporated to further optimize the semantic alignment between different modalities. Experimental results demonstrate the superior performance of Ta-CLIP across several Chinese cross-modal retrieval datasets, highlighting its significant potential for real-world applications.