A High-Accuracy and Low-Resource Design for Nonlinear Functions in AI Accelerators

In deep learning models, the increasing complexity of activation functions makes hardware optimization crucial. This paper proposes an efficient computation framework for 4 elementary functions (exponential, logarithmic, reciprocal and square root). By leveraging the floating-point representation and employing piecewise linear (PWL) approximation for the mantissa, our method substantially reduces multiplication operations and storage overhead. Our framework supports the execution of prevalent activation functions through combinations of these functions, further enhancing its versatility. The proposed design is validated on Xilinx Zynq X7CZ020 FPGA. Specifically, the FP16 4-segment pipelined module achieves a mean relative error of 2.26e-3 across all valid input ranges, but only utilizes 725 LUTs and 129 FFs without using DSP. It reaches a maximum working frequency of 163.64 MHz, and requires a 4 clock cycles latency, demonstrating significant advantages in resource utilization and computation efficiency.