Progressive Distillation With Optimal Transport for Federated Incomplete Multi-Modal Learning of Brain Tumor Segmentation

Multi-modal Magnetic Resonance Imaging (MRI) provide sufficient complementary information for brain tumor segmentation, however, most current approaches rely on complete modalities and may collapse with incomplete modalities. Moreover, most existing endeavors focus on training with centralized databases, failing to make full use of distributed multi-silo datasets with rich patient data to learn a more robust brain tumor segmentation model. In this paper, considering the distributed training scenarios, we formulate Federated Incomplete Multi-modal Learning (FedIML) for brain tumor segmentation, and propose Progressive distiLlation with Optimal Transport (PLOT) framework to gradually train a modality robust segmentation model at each client and achieve compatible model aggregation at the server. Specifically, to remedy the issue of unstable local training caused by the random modality input, we present Modality Progressive Distillation (MPD), a multi-level knowledge distillation strategy guided by a modality routing mechanism. At each client, MPD provides a gradually learning course for a student model in an easy-to-hard manner to achieve a stable local training process. Moreover, to address the problem that the layer-wise knowledge from different models may contradict, at the server, we design Optimal Transport-guided Model Aggregation (OTMA) strategy, which yields a global alignment solution for model parameters via solving an optimal transport problem. OTMA can achieve a compatible parameter aggregation and boost the distributed training. Extensive experiments on the BraTS-2021 dataset demonstrate the effectiveness of the proposed framework over state-of-the-art methods.