A Flexible and High-Precision Activation Function Unit Based on Equi-Error Partitioning Algorithm

The diversity of activation functions has gradually increased to accommodate different tasks in modern deep neural networks (DNNs). However, these novel activation functions involve more nonlinear operations relative to traditional activation functions, which increases the computational complexity. To address these issues, a piecewise linear (PWL) approximation algorithm called Equi-Error Partitioning Algorithm is proposed in this paper. The algorithm aims at balancing the errors between segments and solves the problem of excessive precision that exists in other PWL approximation methods and achieves on average 30.07× better mean squared error compared to the previous works. Based on this algorithm, we propose an activation function unit (AFU) which enables the addressing scheme of non-uniform segments and provides reconfigurability for all common activation functions by reloading parameters. End-to-end evaluation with several DNNs shows the accuracy loss is all less than 0.06% with 64 segments.