Text Instructed Zero-Shot Object Counting with Spatial Prior Guidance Network

The majority of existing counting models are designed to operate on a singular object category, for instance, pedestrians or vehicles. The rise of multi-modality base networks, such as CLIP (Contrastive Language-Image Pre-training), has paved the way for object counting. This approach facilitates the counting of objects across diverse classeswithin one image by leveraging textual prompts. However, some counting models based on CLIP confront a primary challenge. Specially, the CLIP model exhibits limited sensitivity towards location information, which prioritizes global content over the precise localization of objects. Consequently, directly employing the CLIP model is regarded as suboptimal. To address the sensitivity issue of object position in multimodal models based on CLIP in object counting, we propose a spatial prior guidance (SPG) network in this paper. The spatial prior guidance module attempts to extract spatial positions for guiding the vision encoder. Thus, it concentrates on the specific object regions. Experimental results on FSC-147 and ShanghaiTech benchmark datasets demonstrate that our approach outperforms other methods.