A Novel Sigmoid Function Approximation Suitable for Neural Networks on FPGA

Artificial Neural Networks (ANN) is invading a lot of practical applications in our life nowadays. One of the main blocks of ANN is the activation function block, which is based on the sigmoid function. The hardware implementation of sigmoid function is a challenging task; hence some approximation techniques were previously developed. In this paper, a novel sigmoid approximation technique is proposed and compared with previous techniques, on both simulation and hardware design levels. They are applied in a neural network application, where the proposed technique showed high accuracy compared to the original sigmoid function. Moreover, the different techniques are implemented on Virtex 7 FPGA using IEEE 754 Floating Point representation to achieve high precision, where the proposed approximation consumed the least hardware area utilization compared to previous works for clock frequency of 358.166 MHZ.