Accelerating Kernel Ridge Regression with Conjugate Gradient Method for large-scale data using FPGA High-level Synthesis

In this work, we accelerate the Kernel Ridge Regression algorithm on an FPGA-based adaptive computing platform to achieve higher performance within faster development time by employing a design approach using high-level synthesis (HLS). We partition the overall algorithm into a quadratic complexity part that runs on the FPGA fabric and a linear complexity part that runs in Python on the ARM processors. In order to avoid storing the potentially huge kernel matrix in external memory, the designed accelerator computes the matrix on-the-fly in each iteration. Moreover, we overcome the memory bandwidth limitation by partitioning the kernel matrix into smaller tiles that are pre-fetched to small local memories and reused multiple times. The design is also parallelized and fully pipelined. The final accelerator can be used for any large-scale data without kernel matrix storage limitations and with an arbitrary number of features. The accelerator reaches 86 GFLOPS on a Kria XCK26 FPGA and 231 GFLOPS on an Alveo U30 card which is within 72% and 90% of the estimated peak performance of those boards. We use the Pynq framework to directly call the accelerator from our Python code. This work is an important first step towards a library for accelerating different Kernel methods for Machine Learning applications for different FPGA platforms that can be used conveniently from Python with a NumPy-like interface.