A Network Coding-Based Approach to Floating-Point Sum Reduction

Floating-point sum reduction is a key primitive for collective communication operations such as All-Reduce and Reduce-Scatter. Existing reduction schemes typically use simple topologies, like trees or lines. In this paper, we discuss floatingpoint sum reduction over a general network topology represented by a directed acyclic graph with fixed link capacities. In this network, a destination node seeks to compute the sum of real-valued inputs from multiple sources using floating-point arithmetic. We propose a network coding-based approach to address this problem, aiming to achieve both high computational throughput and accuracy. By leveraging a dual relationship between multicast and sum reduction, we show that real number network coding achieves the theoretical max-flow rate. To address the unique challenges of floating-point arithmetic, we analyze rounding errors in encoding and decoding operations and derive error bounds. Based on the error bounds, we develop an optimization framework for constructing high-accuracy network codes.