Federated Multimodal Learning for IoT Applications: A Contrastive Learning Approach

The increasing use of IoT devices has led to the generation of vast amounts of data from various modalities, making them ideal candidates for federated learning (FL). FL is a machine learning approach that allows models to be trained on decentralized data sources without compromising data privacy and security, making it a suitable technique for IoT applications. However, existing FL methods mainly focus on unimodal data, which limits their applicability in real-world IoT applications where devices consist of data from multiple sources. To address this limitation, we propose a Federated Multimodal Learning approach for IoT applications with a dual contrastive regularization (DC-MMFed). Our proposed method enables clients to learn a joint multimodal representation from multimodal data while preserving data privacy. By using contrastive learning, our method allows for learning discriminative features across different modalities. We evaluate our approach on a human activity recognition dataset and demonstrate its superior performance on different downstream tasks compared to baseline FL methods. Our work contributes to privacy-preserving multimodal machine learning in IoT applications, advancing network management without compromising data security. By leveraging DC-MMFed, devices can perform more accurate and robust machine learning tasks without data centralization or sharing, thus maintaining data privacy and security.