Multimodal One-class Learning for Malicious Online Content Detection

Social media content can present a number of threats, including misinformation and hate speech towards specific demographic groups. One challenge is to effectively discriminate between benign and malicious posts, given the massive amount of available content. In this context, predictive models for malicious content detection can be extremely valuable, leading to the automatic removal of posts and user accounts or content being flagged for subsequent moderation. However, some of the existing detection models are limited to the analysis of a single data modality. At the same time, most multi-modal approaches operate in a fully supervised learning setting that assumes the availability of labeled data for both benign and malicious content. In this paper, we fill this gap by proposing a multimodal one-class learning approach for malicious online content detection. Our approach leverages feature extraction, dimensionality reduction, and one-class learning models to analyze text and image data in online posts simultaneously. Models learn their decision function in the challenging scenario where only benign online content is used as training data, overcoming the limitations of a fully supervised setting. Our experiments with two real-world datasets containing misinformation and hate speech posts reveal the effectiveness of different combinations of one-class learning models and dimensionality reduction techniques.