Attribute-enhanced Selection of Proper Demonstrations for Harmful Meme Detection

Internet memes are becoming increasingly prevalent across social media platforms, but also leveraged by malicious users to spread harmful speech. Recent studies have achieved remarkable improvement on detecting harmful multi-modal memes, but they neglect the great impact of good demonstrations on the task. In this paper we take a closer look at prompt learning for harmful meme detection. We argue that, for a given meme sample, its good demonstrations should not only share similar embeddings of text and images but also attack attributes like races and regions. We propose a novel method of attribute-enhanced prompt learning to retrieve and rank demonstrations from the training dataset, considering both embedding similarities in each modality and attack attributes. We conducted comprehensive experiments including overall performance and various demonstration strategies. Experimental results on two widely-used datasets show that our model achieves state-of-the-art results and the effectiveness of our demonstration strategies.