HaLo-FL: Hardware-Aware Low-Precision Federated Learning

Applications of federated learning involve devices with extremely limited computational resources and often with considerable heterogeneity in terms of energy efficiency, latency tolerance, and hardware area. Although low-precision training methods have demonstrated effectiveness in accommodating the device constraints in a centralized setting, their applicability in distributed learning scenarios featuring heterogeneous client capabilities has not been well explored. In this work, we design a hardware-aware low-precision federated training framework (HaLo- FL) tailored to heterogeneous resource-constrained de-vices. In particular, we optimize the precision for weights, activations, and errors for each client's hardware constraint using a precision selector (named HaLo-PS). To validate our approach, we propose HaLoSim, a hardware evaluation platform that enables precision reconfigurability and evaluates hardware metrics like energy, latency, and area utilization on a crossbar-based In-memory Computing (IMC) platform.

I. Introduction

The advent of Federated Learning (FL) has facilitated the training of models on a large scale, leveraging distributed data sources [1]. FL has demonstrated effectiveness in various application domains, including healthcare [2], [3], IoT [4], [5], autonomous driving [6], [7] among others. Given that FL is deployed in resource-constrained devices, often each with different constraints and priorities, it becomes imperative to address the specific training and inference requirements of the client devices [8]. For example, a drone will have energy constraints, whereas a smartwatch will be constrained in terms of area.