Multimodal Adversarial Defense Trained on Features Extracted from Images and Brain Activity

Deep neural networks (DNNs) have achieved significant advancements in artificial intelligence (AI) but remain vulnerable to adversarial attacks. In contrast, the human cognitive system, integrating multimodal processing and brain activity, shows high robustness against such attacks. This paper introduces a novel multimodal adversarial defense method that incorporates brain activity data to enhance the DNN robustness. We use functional magnetic resonance imaging (fMRI) data recorded while subjects view images and combine it with visual data to create a more robust classifier. Experimental results using the NSD dataset demonstrate that our method improves robustness against adversarial attacks compared to traditional methods. This research underscores the potential of leveraging brain activity data to develop more robust AI systems.