Efficient activation functions for the back-propagation neural network

Summary form only given. A new family of activation functions for the back-propagation algorithm has been proposed, whose derivatives belong to the Sech/sup n/ (x) family for n=1,2, . . .. The maximum value of the derivatives varies from 0.637 to 1.875 for n=1-6, and thus a member of the activation function family can be selected to suit the problem. Results of using this family of activation functions show orders of magnitude savings in computation. A discrete version of these functions was also proposed for efficient implementation. For the parity 8 problem with 16 hidden units, the new activation function f/sub 3/ uses 300 epochs for learning as compared to 500000 epochs used by the standard activation function.<
>