A Multimodal Content Moderation System using Adversarial Machine Learning

Memes are a visual representation of images with text embedded in them that conveys the thoughts and feelings of a peculiar audience and is mainly intended to elicit humor. It is widely spread across social media platforms in the guise of sardonic images, humorous jokes, and several other viral sensations. After the profound success of the late fusion technique that combines multimodal features, researchers have used stacked LSTM to extract textual features, whereas VGG16 has been used to extract visual features. Apart from this, BiLSTM and CNN have been used to extract contextual features. This study suggests a unique deep learning-based method for classifying objectionable memes within a multimodal dataset to address this problem. The proposed system uses machine learning to train the classifier on labeled data, which allows it to categorize the information accurately. The proposed system uses both text and image data. To improve user experience and safety, offensive images are blurred, and offensive words are shuffled using adversarial training.