PromptLandscape: Guiding Prompts Exploration and Analysis with Visualization

Prompts, which are pieces of text injected into the inputs of NLP models, can significantly enhance these models’ performance. Despite their effectiveness, a straightforward method for exploring a large number of possible prompts and assessing their quality in down-stream tasks is lacking. This paper introduces PromptLandscape, a visualization approach to exploring prompts in the zero-shot image classification context. Our focus is on the CLIP model [26], which has been pre-trained on a large number of image-text pairs and exhibits exceptional performance in zero-shot image classifications. However, its performance is also considerably influenced by the text labels. Prompts were introduced to refine the text labels and improve the classification performance. Nevertheless, the prompts are often proposed through trial-and-error and it is challenging to find the best prompts for different image datasets. PromptLandscape presents the large number of possible prompts as points in a 3D space to guide their exploration and enables users to decode any points back as texts for better interpretation. Through concrete case studies, we demonstrate the effectiveness of PromptLandscape in (1) finding the best prompts for image classifications and (2) interpreting the prompts to reveal the crucial features of the studied image datasets.