Divide and Conquer: The First Step Towards Adaptable Internet of Models

Enhancing model performance on low-capacity devices remains a significant challenge in the Internet of Models (IoM). Inspired by the efficiency of disentangled representation learning, this study proposes a model decomposition approach to improve response time on low-capacity devices while maintaining accuracy, through collaborative learning with models deployed on higher-capacity devices. First, we introduce a divide-and-conquer strategy that decomposes and learns knowledge within each data sample. This enables lightweight sub-models, tailored to specific knowledge components, to respond efficiently on low-capacity devices in the IoM system. Second, we design a progressive crossfusion mechanism to promote mutual enhancement among these knowledge-specific sub-models. Third, we optimize the learning process by aligning the updates of these sub-models with those of the models on higher-capacity devices. This collaborative optimization is guided via knowledge distillation during model aggregation. Experimental results on image classification tasks demonstrate that our approach reduces response time by 9.87 % to 60.31 % without compromising accuracy.