Reduced-Precision Floating-Point Formats on GPUs for High Performance and Energy Efficient Computation

Most computer programs are written using either or both 32- and 64-bit floating-point formats (FP32 and FP64). To save memory space, improve the computation speed, and improve energy efficiency by eliminating waste bit data in floating-point data, we need reduced-precision formats that can represent various and lower floating-point precisions. This paper proposes the implementation of reduced-precision floatingpoint formats, which provide different and shorter significand lengths compared with the IEEE formats on graphics processing units. This paper demonstrates the performance of some linear algebra kernels (AXPY, GEMV, and CSRMV) using reduced-precision formats on GeForce GTX 980. Our implementations with the reduced-precision formats outperformed the optimized vendor implementations for FP32 and FP64.