Cognitive-Visual Fusion for Target Classification in Rapid SAR Image Series Visual Presentation

The ability of deep learning models in image classification tasks has already suppressed humans. However, The convolutional neural network (CNN) utilized in computer vision tends to focus on the local features of the images, which is different from human perception ability. In this paper, we aim to combine visual capabilities from the machine and cognitive capabilities from humans to perform image classification. In particular, we use the EEG data evoked by sparse Synthetic-aperture radar (SAR) target images presented in rapid series and the stimuli SAR image series to detect the SAR images containing targets. We propose a multimodal framework named Cognitive-Visual Fusion Network (CVF-Net) that employs both visual domain information and cognitive domain information. We demonstrate that the proposed network can effectively detect target images and show that multimodal information has significant improvement over single domain knowledge.