Heuristic Dropout: An Efficient Regularization Method for Medical Image Segmentation Models

For medical image segmentation in a real scenario, the amount of accurate annotation data at the pixel level is typically small, which tends to cause an overfitting problem. This manuscript goes deep into the research of the Dropout algorithm, which is commonly used in neural networks to alleviate the overfitting problem. From the perspective of solving the co-adaptation problem, this manuscript explains the basic principles of the Dropout algorithm and discusses the existing limitations of its derivative methods. Furthermore, we propose a novel Heuristic Dropout algorithm to address these limitations. The proposed algorithm takes information entropy and variance as heuristic rules. It guides our algorithm to drop features suffering from co-adaptation problem more efficiently and thus can better alleviate the overfitting problem of small-scale medical image segmentation datasets. Experiments on medical image segmentation datasets and models show that the proposed algorithm significantly improves the performance of these models.