Clustering-Enhanced Reinforcement Learning for Adaptive Offloading in Resource-Constrained Devices

The Federated Edge Artificial Intelligence (Edge AI) deploys AI applications on Internet-of-Things (IoT) devices, addressing data privacy concerns in the real world. To achieve effective Federated Learning (FL), three challenges must be addressed: i) limited computing power on devices, ii) non-uniform impacts on devices, and iii) adaptability to changing network conditions. This study introduces a new algorithm called Adaptive Offloading Point (AOP), designed to accelerate local training on constrained devices. It decomposes deep neural network (DNN) layer blocks, enabling training on both client and server sides. The novelty of the proposed method lies in using a reinforcement learning-based Gaussian mixture model (GMM) clustering to dynamically determine DNN layer offloading, addressing nullability, computation uniformity, training time, and network bandwidth variation issues. Experimental results on real devices, using vision transformer (ViT) models with an identification image dataset, show that AOP's training time is significantly faster than that of previous baseline methods.