Patch Target Guided Dual-Branch Deep Multiple Instance Learning for 3D MRI Analysis

Deep multiple instance learning (MIL) has attracted considerable attention in medical image analysis, since it only requires image-level labels for model training without using fine-grained (or patch) annotations. Unfortunately, MIL-based methods might lose some significant patch features. Although pseudo-label-based methods, which assign a pre-defined label to each patch, can explore more patch-level features, they might bring label noise and make the patch-level features lose diversity, thereby possibly restricting the model performance. To overcome this issue, we propose a novel gradient-based patch target generation (PTG) module to dynamically produce a feature vector for each patch as its target. Additionally, based on the PTG module, we propose a patch-target guided dual-branch deep MIL framework for 3D MRI data analysis, where both the two branches consist of a CNN model to extract patch-level features, an attention module to interpret the significance of patches, and a bag-level classifier, while the second branch also contains the PTG module to generate patch targets of patches. Moreover, the two branches are alternatively updated in our framework, resulting in a bi-level optimization problem, and thus we design a bi-level optimization algorithm to solve our proposed objective function. Extensive experiments demonstrate the superior classification and interpretation performance of the proposed framework over recent state-of-the-art methods. Codes are available at https://github.com/daimz1213/mil2024.