Estimating Viewed Image Categories from Human Brain Activity via Semi-supervised Fuzzy Discriminative Canonical Correlation Analysis

This paper presents a method to estimate viewed image categories from human brain activity via newly derived semi-supervised fuzzy discriminative canonical correlation analysis (Semi-FDCCA). The proposed method can estimate image categories from functional magnetic resonance imaging (fMRI) activity measured while subjects view images by making fMRI activity and visual features obtained from images comparable through Semi-FDCCA. To realize Semi-FDCCA, we first derive a new supervised CCA called FDCCA that can consider fuzzy class information based on image category similarities obtained from WordNet ontology. Second, we adopt SemiCCA that can utilize additional unpaired visual features in addition to pairs of fMRI activity and visual features in order to prevent overfitting to the limited pairs. Furthermore, Semi-FDCCA can be derived by combining FDCCA with SemiCCA. Experimental results show that Semi-FDCCA enables accurate estimation of viewed image categories.