spiNNlink: FPGA-Based Interconnect for the Million-Core SpiNNaker System

SpiNNaker is a massively-parallel computer system optimized for the simulation, in real time, of very large networks of spiking neurons. The system consists of over 1 million, energy-efficient ARM cores distributed over 57,600 SpiNNaker chips, each of which contains 18 cores interconnected by a neurobiologically-inspired, asynchronous (clock-less) Network-on-Chip. The NoC is extended to the chip boundary for chip-to-chip communication. To construct the massively-parallel system, SpiNNaker boards, housing 48 SpiNNaker chips, are connected together using FPGA-based, high-speed serial links. This paper presents some of the novel aspects of the design and implementation of the bespoke interconnect, including a credit-based, reliable frame transport protocol that allows the multiplexing of asynchronous SpiNNaker channels over the serial links, and an efficient FPGA-to-SpiNNaker chip interface that provides twice the throughput of traditional asynchronous interfaces. SpiNNaker houses 3,600 Xilinx Spartan-6 FPGAs, provides a bisection bandwidth of 480 Gbit/s, and ran the first-ever, true real-time brain cortical simulation [1] - a feat not currently achievable using conventional HPCs or GPUs.