3-way Parallel Fusion of Spatial (sMRI/dMRI) and Spatio-temporal (fMRI) Data with Application to Schizophrenia

Advances in brain imaging acquisition techniques allow data from multiple modalities to be collected from each subject offering different but limited views of the structural, functional or temporal properties of human brain. Multimodal fusion can provide a way to leverage different perspectives from multiple complementary modalities. However, most current fMRI-related multimodal fusion approaches are restricted to the second-level 3D features, rather than the original 4D fMRI data. Here we are motivated to propose a novel 3-way fusion approach that can incorporate temporal information from fMRI by parallelizing “group ICA” and “parallel ICA” under a global optimization. Simulations show that GICA+pICA provides more accurate inter-modality linkage detection under both strong and weak correlations. In real data application, one linked fMRI-sMRI-dMRI component was identified showing differences between schizophrenia and controls in all modalities, demonstrating the stability of GICA+pICA to identify inter-modality linkage among three modalities.