Leveraging Sparsity of SRNNs for Reconfigurable and Resource-Efficient Network-on-Chip

Establishing reconfigurable hardware connectivity in a Spiking Recurrent Neural Network (SRNN) is a challenging task due to the requirement of hardware capability for a fully-connected SRNN. For large-scale SRNNs, the Address Event Representation (AER) packet-based spike communication approaches are popular. However, these packet-based networks are inefficient for low-power, low-area implementations of SRNNs due to additional packet handling circuits. Circuit-switched crossbars address this problem to some extent for small-scale SRNNs, but crossbars are non-scalable and inflexible for NoC implementations due to the multiplicatively increasing hardware requirement as number of nodes increase. Thus, even for sparse SRNNs with low synaptic density, the hardware resource requirement of crossbars remains disproportionately high. In this work, we propose a Spike-routing Circuit-Switched Network, SpiCS-Net (pronounced "spikes-net"), based on Clos topology for leveraging sparse connectivity of SRNNs leading to resource-efficient implementations. SpiCS-Net is highly scalable when compared to crossbars, avoids spike-to-packet conversion of AER-based approaches, and leverages varying degrees of sparseness in SRNNs for resource-efficient implementations. We further introduce the concept of Concurrent Connectivity, which defines the total density of connections that can be established concurrently via a Network-on-Chip architecture. The Concurrent Connectivity of the proposed SpiCS-Net architecture can be tuned for specific synaptic density requirements of an SRNN. We present an in-depth mathematical analysis and design process of SpiCS-Net for sparse SRNN implementations. The proposed network delivers nearly 13.9x savings in power compared to related packet-based approaches in the literature. The throughput of the 128 × 128 SpiCS-Net architecture is 3.6 × 10
⁹
spikes per second. The proposed network also offers up to 21.5x savings in area, which is further reduced by 3.5× on lowering the concurrent connectivity of SpiCS-Net from 100% to 37.5%.