Fully Automatic Bayesian Method for Trainable Activation Function and Deep Neural Networks

In this study, we introduce a novel technique that incorporates an MCMC-based algorithm within a Bayesian framework to estimate the parameters of trainable activation functions and model weights in deep neural networks. This approach aims to enhance network performance by effectively tackling the challenges of parameter learning and reducing the risk of overfitting. Our method leverages an efficient sampling process to accelerate convergence times. The efficacy of the proposed activation function is demonstrated through experiments on two datasets within the remote sensing domain. These experiments reveal that our approach enables neural networks to achieve high levels of accuracy, reaching up to 92%, while keeping the model complexity low. This suggests that the proposed method could offer significant benefits for deep learning applications, particularly in fields requiring precise and reliable predictive modeling.