Co-neighbor multi-view spectral embedding for medical content-based retrieval

Multimodal medical data from various information sources are often used to depict patients. We refer to each source as a `view'. Multi-view features could provide complementary information to each other; thus by fusing the multi-view features, we could greatly enhance the current medical content-based retrieval framework. In this paper, we propose a Co-neighbor Multi-view Spectral Embedding (CMSE) algorithm, which is an advanced feature fusion method based on the multi-view spectral analysis. CMSE aims to seek a smooth embedding for the multi-view features by maximizing the neighborhood affinity across all feature spaces. We evaluated the proposed CMSE algorithm using a freely available neuroimaging database, ADNI, with 331 pre-diagnosed subjects. Totally, 9 views of features were extracted for validation, and an improved retrieval performance was achieved over other state-of-art feature fusion methods.