AQMFL: An Adaptive Quantization Framework for Multi-modal Federated Learning in Heterogeneous Edge Devices

With the wide application of multi-modal fusion sensing in scenarios such as autonomous driving and human-computer interaction, the privacy security and communication burden caused by massive data uploading need to be solved urgently. Federated Learning (FL) has received significant attention as a privacy-preserving distributed machine learning paradigm. Recent Multi-Modal Federated Learning (MMFL) focuses on addressing modal heterogeneity to enhance accuracy and speed up convergence. However, it overlooks the huge communication overhead in updating complex multi-modal network models, especially in edge environments with limited bandwidth. At the same time, the state-of-the-art communication-efficient FL methods are not customized to the MMFL characteristics. In this paper, we propose the Adaptive Quantization framework for Multi-modal Federated Learning (AQMFL). AQMFL implements decision-level multi-modal fusion locally by using parallel training and model ensemble, supporting its adaptation to modal heterogeneity and flexible deployment. AQMFL can adaptively allocate the number of quantization levels of gradient according to the modal contribution and the heterogeneous communication ability of nodes, which speeds up the system convergence and achieves a better balance between accuracy and communication efficiency. Compared with the classical baselines, AQMFL can reduce the total communication overhead by up to 50.47% and the total training time by up to 52.11% while maintaining the accuracy.