Heterogeneous Federated Large-Scale Model Architecture With Privacy Protection for Cross-Corporation Non-i.i.d. Fault Diagnosis

Large-scale models (LMs) and data, as valuable assets of various corporations, are subject to strict privacy protection, posing significant challenges for LM training and application. To address the challenge, a dual-stage heterogeneous federated LM architecture (HFLA) is proposed with an upstream–downstream learning-deployment structure. The architecture enables cross-corporation non-independent and identically distributed (non-i.i.d.) fault diagnosis, while ensuring that models and data remain within their respective private domains. HFLA offers several advantages, including strong heterogeneity, strict privacy protection, low learning cost, and lightweight deployment. In the upstream learning stage, a bidirectional heterogeneous knowledge distillation framework is designed to facilitate learning between the server’s LM and the clients’ small-scale models (SMs), which are adaptively generated from the LM according to the difficulty of tasks. A differential federated aggregation strategy is also proposed to address the heterogeneity of clients’ data and models, as well as the non-i.i.d. task issue, enabling efficient aggregation of LMs. In the downstream deployment stage, the learned LM is compressed via unidirectionally heterogeneous knowledge distillation to task-private lightweight SMs for deployment on multiple corporations. In experiments, HFLA is pretrained using nine non-i.i.d. publicly available datasets and deployed to three completely different corporate terminals: aero-engine intershaft bearings, synchronous motors, and wind turbine planetary gearboxes. The average fault diagnosis accuracy across the three non-i.i.d. terminals under various few- and short-shot conditions reached 93.66% and 98.14%, respectively.