The effect of the activation function of the back propagation algorithm

The effect is investigated of the activation function (the node nonlinearity) on the performance of the backpropagation algorithm in training a multilayer perceptron. Simulations for a two-layer neural net with two input nodes, a single output node, and eight nodes in the hidden layer are presented. The main conclusion presented is that the universally accepted smoothness condition for the nonlinearity is not necessary for the proper functioning of the algorithm.<>