Accelerating parallel data processing using optically tightly coupled FPGAs

A cutting-edge field programmable gate array (FPGA) card can be equipped with high-bandwidth inputs and outputs by high-density optical integration, e.g., onboard Si-photonics transceivers or co-packaged optics. It is possible to implement highly parallel data processing by using tightly coupled FPGAs. We previously proposed a lightweight, fully connected inter-FPGA network called OPTWEB for efficient collective communications. OPTWEB discards the traditional packet communication mechanism and introduces dedicated memory processing to minimize communication overhead. In this paper, we provide the design guidelines for a parallel data application on FPGAs connected by OPTWEB. We then present a generalized diameter-two hypercube network topology for connecting multiple FPGAs, which enables larger applications. In the guidelines, data to be exchanged should be directly processed by the dedicated memory on each FPGA, and parallelism should be implemented to match the collective communications provided by OPTWEB. We illustrate two case studies, one on parallel counting sort and the other on sharing averaged parameters, on eight custom Stratix10 MX2100 FPGA cards with 800 Gbps network bandwidth. The parallel counting sort operates up to 5.3 times faster than a comparable CPU cluster with the conventional InfiniBand network, and with sharing averaged parameters, up to 250 times faster operation is achieved. The two applications utilize a total of 80.1% and 60.6% of the adaptive logic modules of the FPGA, respectively.