EBACA: Efficient Bfloat16-based Activation Function Implementation Using Enhanced CORDIC Architecture

With the rising demand for Convolution Neural Network (CNN) inference models on edge devices, there is a need for power, performance, and area-efficient design. In this work, we present an ultra-low-power, area-efficient Bfloat16 (BF16)-based floating-point implementations for activation functions using an improved Coordinate-Rotation-Digital-Computer (CORDIC) algorithm. Three diverse design methodologies in the form of serial, parallel, and hybrid architectures are presented for the hardware implementation of Softmax functions, and one CORDIC-based BF16 architecture each is designed for achieving tanh, and Sigmoid functionality in hardware. A comprehensive comparison of the proposed BF16-based CORDIC implementations against current state-of-the-art (SOTA) designs supporting half-precision floating-point (FP16), and single-precision floating-point (FP32) data-formats are presented. The proposed designs are realized on both FPGA and ASIC at 100 MHz and 200 MHz respectively along with SOTA designs for benchmarking. In ASIC implementation, BF16-based CORDIC resulted in PPA gains of $1.24 \times, 1.81 \times$ for tanh and Sigmoid functions over FP16 design, and 11.24 $\times$, $26.76 \times$ over FP32 methods respectively. Similarly, significant PPA improvements was reported for BF16-based serial, parallel, and hybrid Softmax function implementations over FP16 and FP32. The proposed ultra-low power and area-efficient BF16based implementation for activation functions is a step towards establishing highly efficient hardware CNN accelerators.