Multimodal Graph Convolutional Network on Brain Structure and Function in Adolescent Anxiety and Depression

Multimodal analysis of Magnetic Resonance Imaging (MRI) data enables leveraging complementary information across multiple imaging modalities that may be incomplete when using a single modality. For brain connectivity analysis, graph-based methods, such as graph signal processing, are effective for capturing topological characteristics of the brain structure while incorporating neural activity signals. However, for tasks like group classification, these methods often rely on traditional machine learning algorithms, which may not fully exploit the underlying graph topology. Recently, Graph Convolutional Networks (GCN) have emerged as a powerful tool in brain connectivity research, uncovering complex nonlinear relationships within the data. Here, we develop a multimodal GCN model to jointly model brain structure and function to classify anxiety and depression in adolescents using the Boston Adolescent Neuroimaging of Depression and Anxiety dataset. The graph’s topology is initialized from structural connectivity derived from diffusion MRI, while functional connectivity is incorporated as node features to improve distinction between anxious, depressed patients and healthy controls. Interpretation of key brain regions contributing to classification is enabled through Gradient-weighted Class Activation Mapping, revealing the influence of the frontal and limbic lobes in the diagnosis of the conditions, which aligns with previous findings in the literature. By comparing classification results and the most discriminative features between multimodal and unimodal GCN-based approaches, we demonstrate that our framework improves accuracy in most classification tasks and reveals significant patterns of brain alterations associated with anxiety and depression.