A Generic and Efficient Communication Framework for Message-Level In-Network Computing

Message-Ievel in-network computing (MINC) emerges as a promising hardware acceleration method that utilizes accelerators to offload message-level computation and enhance application performance in the datacenter. However, the development of MIN C applications is challenging in the communication aspect due to poor portability and under-utilized resource. In this paper, we present Leo, a generic and efficient commu-nication framework for MINC. Leo facilitates portability across both application and hardware via introducing a communication path abstraction, which is capable of describing generic applications with predictable communication performance across diverse hardware. It further incorporates a built-in multi-path communication over CPU and accelerator to enhance communication effi-ciency. We have implemented a prototype of Leo and evaluated it with four case studies on testbeds covering FPGA-based, SoC-based smartNICs and GPU. Experiments show that Leo achieves genericity and efficiency across MINC applications, yielding 1.2-4.7 x speedup over baselines with negligible overhead.