SQNL: A New Computationally Efficient Activation Function

A new activation function is proposed. This activation function uses the square operator to introduce the required non-linearity as compared with the use of an exponential term in the popular TanSig. Smaller computational operation count characterizes the proposed activation function. The key to the effectiveness of this function is a faster convergence when used in Multilayer Perceptron Artificial Neural Network architectural problems. Besides, the derivative of the function is linear, resulting in a quicker gradient computation. The effectiveness and efficiency of the proposed activation function have been compared with the TanSig and the computationally efficient ElliotSig functions using selected UCI datasets. The MNIST handwritten dataset was also used to show the ability of this function on more massive datasets. An empirical comparison suggests that the proposed function outperforms TanSig and ElliotSig in convergence time as well as in generalization metrics for most datasets.