Hierarchical and Decentralised Federated Learning

Federated Learning (FL) is a recent approach for distributed Machine Learning (ML) where data are never communicated to a central node. Instead, an ML model (for example, a deep neural network) is initialized by a designated central (aggregation) node and shared with training nodes that have direct access to data of interest. These training nodes then perform small batches of training on their local data. Periodically, each training node submits ML model parameter/weight updates to the central node. The central node aggregates the parameters/weights to create a new global ML model that it then re-shares with the training nodes. This process can either take place indefinitely or be repeated until the ML model converges with respect to some evaluation metric (for example, mean average error, accuracy).