Refining Multi-Teacher Distillation for Multi-Modality COVID-19 Medical Images: Make the Best Decision

Accurate COVID-19 lesion segmentation is vital for diagnosing lung infections. Experiment results suggest that a great teacher may not produce a brilliant student due to the huge gap in contextual semantics. Also studying from a single knowledge domain does not improve the student's learning ability. To help student understand knowledge in a rational manner and solve the aforementioned issues we creatively design a multi-teacher distillation framework based on the CNN-Transformer. Our proposed framework has three main benefits 1) We design GMDiff (Generate Medical Diffusion Model) to solve the scarcity problem of high-quality COVID-19 medical images to adapt few-shot learning. 2) The DWD (Dynamic Weight Distribution) can adaptively adjusts the weight of a weak teacher to reduce incorrect guidance on student learning. When the weak teacher has enough confidence in their decisions the students can achieve satisfactory results. 3)To bridge the semantic gap between the student and multi-teacher and facilitate interaction we propose the MSL (Multi-Scale Loss) AGL (Attention Gradient Loss) and EPL (Edge Pixel Loss) strategies to supervise student learning. Before utilizing the above strategies we adopt IPM (Information Processing Module) to standardize the feature representation shared by student and teachers. Finally we fine-tune on local details. Extensive experiments show that our proposed distillation framework achieves a state-of-the-art segmentation result on the multi-modality dataset. It may be regarded as a novel baseline for future researches.