MedTimeSplit: Continual dataset partitioning to mimic real-world settings for federated learning on Non-IID medical image data

Traditional Deep Learning (DL) approaches for medical image classification rely on centralized, static datasets, which do not adequately reflect the dynamic, real-world medical practice where data is continually generated. In contrast, Federated Learning (FL) enables decentralized model training on localized data while preserving privacy. Yet, current FL methods struggle to handle Non-Identically Independently Distributed (Non-IID) data streams over time. This paper introduces Med-TimeSplit, a novel dataset partitioning strategy that integrates Online Continual Learning (OCL) with FL to simulate real-world medical data flows more realistically and effectively. MedTimeSplit partitions data into Non-IID, time-based increments, mimicking dynamic sourcing in medical environments. We evaluate its impact on FL model performance for medical image classification, focusing on skin lesions, and analyze the system’s resilience to backdoor attacks. Our experiments demonstrate that MedTimeSplit outperforms existing methods in both accuracy and robustness, offering a viable solution for real-world medical applications. Additionally, we propose new metrics to measure model behavior over time, including Average Bad Decisions (ABD) and Overall Changing Mistakes (OCM), which provide deeper insights into model performance specifically under OCL conditions. The results highlight the promise of combining OCL with FL in the medical domain, paving the way for more secure and adaptive healthcare solutions. The source code is available on GitHub¹.