Moving Compute towards Data in Heterogeneous multi-FPGA Clusters using Partial Reconfiguration and I/O Virtualisation

To improve energy efficiency in data centres, in particular when targeting computing with large data-sets, it is vital to perform processing close to the data instead of moving data to compute due to the energy consumption for network usage. For FPGA-based data centres, this is an opportunity to use partial reconfiguration (PR) for moving compute to data. However, to take advantage of partial reconfiguration, users face design challenges induced by the FPGA vendor PR design flows, which require an accelerator to be compiled for each PR region or shell individually. This is an issue in data centres where different FPGA I/O layouts add a level of heterogeneity to the system. This paper introduces the concept of I/O virtualisation together with remote reconfiguration and shell-based FPGA virtualisation. This allows deploying the same bitstream on FPGAs with different I/O layouts over a network while supporting a modular design flow for high design productivity. Further, our system provides a custom configuration controller that is 8× faster than the Xilinx default configuration solution. A case study on an oil-reservoir algorithm shows a reduction in compilation time by 6×, compared to the Xilinx PR flow. Ultimately, PR reduces the data movement overhead by 38× and helps to solve the same problem with 3.7× less energy on a test platform consisting of 64 Xilinx ZU9EG Zynq MPSoCs and 1 TB DRAM in total.