Semi-Federated Learning for Connected Intelligence With Computing-Heterogeneous Devices

Federated learning (FL) is a promising distributed learning approach which enables multiple devices to collaboratively train deep neural networks in a privacy-preserving fashion. However, computing-limited devices in Internet of Things (IoT) networks are difficult to perform local model training, which is an obstacle to implementing connected intelligence at the network edge. In this paper, we propose a semi-federated learning (SemiFL) framework to support computing-heterogeneous devices for collaborative model training, even if some local devices do not have computing capabilities. Specifically, in SemiFL, computing-limited devices upload raw data to the base station (BS), while computing-powerful devices upload model parameters to the BS. To characterize the achievable learning performance of SemiFL in practical IoT networks, we derive the convergence upper bound with non-independent and identically distributed (non-IID) data. Furthermore, we formulate an optimization problem to minimize the computation and communication latency by jointly design the transmit power of IoT devices and the receive factor at the BS. Accordingly, we propose an two-stage optimization algorithm to solve the formulated non-convex problem efficiently. Extensive simulation results show that our SemiFL framework outperforms traditional FL in both IID and non-IID settings. In addition, it is shown that the proposed algorithm achieves lower latency than benchmarks under different settings.