Learnable Prompts With Neighbor-Aware Correction for Text-Based Person Search

Text-based Person Search (TBPS) is a critical task in multimodal retrieval that seeks to identify target individuals in an image gallery using textual descriptions. The core challenge lies in establishing robust cross-modal alignment to reconcile the inherent heterogeneity between textual and visual representations. While recent advances leverage large-scale vision-language pretrained models (e.g., CLIP and ALBEF) to embed both modalities into a shared latent space, they often prioritize global feature alignment, neglecting localized interactions between textual tokens and image regions. To address these limitations, we propose the Learnable Prompts with Neighbor-aware Correction (LPNC) framework, which aligns identity-specific semantics across modalities while resolving fine-grained domain discrepancies. Specifically, we design a Learnable Prompt-guided Semantic Alignment (LPSA) module that dynamically fuses visual region features and textual token embeddings into a unified pseudo-token within a language-conditioned latent space. By introducing a cross-modal attention mechanism, LPSA enforces part-aware consistency, i.e., localized interactions between noun phrases and their corresponding visual patches, while suppressing modality-specific noise. To mitigate retrieval bias, we further propose the k-nearest Neighbor Correction (kNC), a training-free strategy that calibrates initial scores by leveraging local neighborhood relationships in a reference embedding space, ensuring balanced accuracy. Extensive experiments on three widely-used TBPS datasets demonstrate the superiority of the proposed LPNC method. Notably, our method exhibits consistent performance gains, demonstrating robust improvements of 1%–4% in Rank-1 accuracy compared to competing methods on various datasets. The code is available at https://github.com/DrLazywh/LPNC