A Parameter-Free Hybrid Activation Function for Deep Artificial Neural Network used in Medical Image Segmentation

The efficacy and reliability of segmentation models heavily rely on the activation functions employed within deep learning architectures. However, a persistent challenge lies in the inability of current activation functions to effectively tackle issues like the vanishing gradient problem. To address this concern, a novel activation function tailored specifically for medical image segmentation tasks is proposed. Named Med-ReLU, this innovative function amalgamates the advantageous properties of ReLU and Softsign to foster stable training dynamics while mitigating the risk of dead neurons and gradient loss. Rigorous empirical evaluations highlight Med-ReLU’s superior performance vis-à-vis state-of-the-art activation functions, underscoring its versatility across diverse neural network architectures. The training performance and segmentation accuracy of Med-ReLU have been investigated, demonstrating stable training and resilience against overfitting. In extensive experiments using 2D cell and 3D knee image datasets, Med-ReLU demonstrated its effectiveness with a Dice Similarity Coefficient (DSC) of 0.9529 in 2D validation and 0.8640 in 3D validation, outperforming traditional functions like ReLU and LReLU. In model testing, Med-ReLU achieved an IoU of 0.9149 and DSC of 0.9604 for the 2D dataset, and an IoU of 0.8878 and DSC of 0.9309 for the 3D dataset. Offering a parameter-free alternative, Med-ReLU not only ensures ease of implementation but also holds promise for enhanced detection capabilities within the realm of deep learning image segmentation tasks. This article thus underscores the pivotal role of tailored activation functions in advancing the robustness and efficiency of deep learning applications in medical imaging.