RNS Hardware Matrix Multiplier for High Precision Neural Network Acceleration: "RNS TPU"

An arithmetic circuit is introduced that performs efficient matrix multiplication for hardware acceleration of neural networks, machine learning, web search and other applications. The circuit is comprised of a plurality of two-dimensional systolic arrays of multiplier-accumulators; these systolic arrays are connected to form a hardware matrix multiplier which processes data using high precision, fixed-point RNS arithmetic. The matrix multiplier takes advantage of the carry-free properties of RNS arithmetic by partitioning each RNS digit into its own dedicated matrix multiplier unit. By operating a plurality of digit matrix multipliers in parallel, a suitable RNS word size is realized. Each partitioned digit matrix multiplier processes a plurality of product summations, or dot products. After dot product summation is complete, the matrix multiplier transfers each summation to a normalization pipeline unit. The systolic architecture favors a narrow RNS digit encoding, so each digit matrix multiplier is realized in minimal IC area, using fast adders and multipliers. These features and others result in greater speed and efficiency versus binary fixed-point arithmetic, especially when comparing high precision implementations. Using an FPGA based implementation and analysis, the RNS matrix multiplier performs matrix multiplication of 32.32 fixed-point arithmetic 7 to 9 times more efficiently than a binary matrix multiplier provided the dimension of the matrices multiplied are sufficiently large.