KANB: A Family of Parametric Activation Functions With Trainable Parameters

In this paper, a new family of four-parameter activation functions, referred to as KANB, is introduced to improve the performance of artificial neural networks (ANNs). The activation functions have been designed and tested under various problem settings –covering various ANN architectures, loss functions and datasets– and have succeeded in enhancing performance compared to the well-established ReLU, Sigmoid and Tanh. Initially, the parameters of the activation functions were considered fixed and preset. Later, they were treated as trainable parameters. The comparison of the outcomes revealed better results in the latter case. More specifically, KANB3 with trainable parameters shows strong potential on complex tasks, occasionally surpassing standard baselines by notable margins.