Distributed Generative Model: A Data Synthesizing Framework for Multi-Source Heterogeneous Data

Recent advancement in generative AI influenced a broad area with successful applications across multiple domains, including computer vision, natural language processing, and the Internet of Things (IoT). However, many existing implementations rely on centralized architectures, which introduce security and privacy concerns while also increasing communication overhead. Limited research has explored the development of distributed generative models, particularly in scenarios where training data originates from various heterogeneous sources. To fill the gap, this paper introduces a distributed generative model framework aimed at enhancing data generation in hierarchical IoT systems. The proposed framework supports distributed data generation across three distinct scenarios: feature-related data, label-related data, and feature-label non-related data. Furthermore, both synchronous and asynchronous update mechanisms are incorporated to accommodate diverse application requirements within IoT environments. Comprehensive experiments using simulated, image, and tabular datasets are conducted to assess the performance of the proposed framework in comparison with state-of-the-art methods. The results indicate that the framework effectively produces high-quality synthetic data while preserving the integrity of downstream tasks. Beyond large language models (LLMs), these findings suggest that generative AI have the potential to transform data generation in distributed IoT systems and be extended to a broader range of applications.