Linking Neural Activity to Image Stimuli Through Convolutional Neural Networks: A Methodology

In this study, we explored the brain's response to different visual stimuli using convolutional neural networks (CNNs) and functional magnetic resonance imaging (fMRI). We employed four pre-trained CNNs (IncepV3, ResNet50, VGG16, AlexNet) to extract features from images of four visual stimulus types (Body, Face, Place, Tool) used in the HCP working memory experiment. We calculated the cosine similarity of these image features, and the cosine similarity of the brain's fMRI neural activity during stimulus presentation. Correlation results indicate that different types of visual stimuli elicit specific responses in distinct brain regions, and the bilateral fusiform gyrus is capable of distinguishing all types of visual stimuli, enhancing our understanding of the brain's response to visual stimuli and its relation to working memory.