Test-Time Prompt Tuning for Compositional Zero-Shot Learning

In this paper, we explore Compositional Zero-Shot Learning (CZSL), focusing on the recognition of new attribute-object combinations using existing concepts. Instead of relying on hand-engineered prompts, recent works learn soft prompts for vision language model (CLIP) using the training data. While the learning procedure is parameter-efficient, it is no enough to reduce the distribute difference between training set and test set in CZSL tasks. Meanwhile, the model needs to focus more on the detailed attribute information and the key object within the image. We propose to tuning the learnable parameters of the model in test time, in order to adapt model to test dataset images. Specifically, We perform multi-view augmentation on the test images, and then obtain multiple prediction probability distributions. By using selection filter and minimize the entropy of probability distribution, we can optimise the learnable prompt of the model and perform adaptive learning during the test phase. Extensive experiments on three widely-used dataset and performance improvement on two state-of-the-art methods demonstrate the effectiveness of our plug-and-play approach.