Resource Rationing for Federated Learning with Reinforcement Learning

Federated learning (FL) aims to complete certain machine learning tasks by multiple rounds of progressive learning. Because of this unique characteristic, the importance of each round towards the final model performance varies as the global model converges. Motivated by this, this paper studies how to allocate the quantization bits across rounds. In particular, we propose a new quantization bits allocation mechanism based on deep Q-learning to realize the full potential of “later-is-better” principle. Unlike the existing allocation mechanism, this mechanism learns resources allocation on the fly to maximize the validation accuracy of the global model in the FL while satisfying the resource constraints. Through extended experiments, we concluded that allocating quantization bits by the proposed approaches can indeed improve the training time and final performance of FL compared with the existing resource allocation strategies. We also pointed out the available number of quantization bits and the transmission mode of FL have a significant impact on the performance of quantization bits allocation. The allocation of quantization bits across rounds based on deep Q-learning shows the great potential of resource allocation in improving the performance of FL. Besides, the method is quite general and it can be well extended to other resource rationing problems in FL.