Abstract:
Federated learning (FL) is a promising distributed machine learning scheme where multiple clients collaborate by sharing a common learning model while maintaining their p...Show MoreMetadata
Abstract:
Federated learning (FL) is a promising distributed machine learning scheme where multiple clients collaborate by sharing a common learning model while maintaining their private data locally. It can be applied to a lot of applications, e.g., training an automatic driving system by the perception of multiple vehicles. However, some clients may join the training system dynamically, which affects the stability and accuracy of the learning system a IoT. Meanwhile, data heterogeneity in the FL system exacerbates the above problem further due to imbalanced data distribution. To solve the above problems, we propose a novel FL framework named FedREM (Retain-Expansion and Matching), which guides clients training models by two mechanisms. They are 1) a Retain-Expansion mechanism that can let clients perform local training and extract data characteristics automatically during the training and 2) a Matching mechanism that can ensure new clients quickly adapt to the global model based on matching their data characteristics and adjusting the model accordingly. Results of extensive experiments verify that our FedREM outperforms various baselines in terms of model accuracy, communication efficiency, and system robustness.
Published in: IEEE Transactions on Parallel and Distributed Systems ( Volume: 35, Issue: 7, July 2024)
Funding Agency:
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Guangzhou, China
Linsi Lan received the bachelor's degree in telecommunication engineering from the Minzu University of China, Beijing, China, in 2021. She is currently working toward the master's degree with the School of Intelligent Systems Engineering, Sun Yat-sen University, Guangzhou, China. Her research interests include federated learning, internet of vehicles and autonomous driving.
Linsi Lan received the bachelor's degree in telecommunication engineering from the Minzu University of China, Beijing, China, in 2021. She is currently working toward the master's degree with the School of Intelligent Systems Engineering, Sun Yat-sen University, Guangzhou, China. Her research interests include federated learning, internet of vehicles and autonomous driving.View more
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Guangzhou, China
Junbo Wang (Member, IEEE) received the BS and MS degrees in electrical and electronic engineering from Yanshan University, China, and the PhD degree in computer science and engineering from the University of Aizu, Japan, in 2011. He was a postdoctoral scholar and associate professor with the Univeristy of Aizu, Japan. Currently he is an associate professor with the School of Intelligent Systems Engineering, Sun Yat-sen Un...Show More
Junbo Wang (Member, IEEE) received the BS and MS degrees in electrical and electronic engineering from Yanshan University, China, and the PhD degree in computer science and engineering from the University of Aizu, Japan, in 2011. He was a postdoctoral scholar and associate professor with the Univeristy of Aizu, Japan. Currently he is an associate professor with the School of Intelligent Systems Engineering, Sun Yat-sen Un...View more
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Guangzhou, China
Zhi Li received the bachelor's degree from the School of Traffic and Transportation, Beijing Jiaotong University, in 2019. She is now working toward the PhD degree with the School of Intelligent Systems Engineering, Sun Yat-sen University. Her current research interests include intelligent transportation system, multi-agent reinforcement learning and graph neural networks.
Zhi Li received the bachelor's degree from the School of Traffic and Transportation, Beijing Jiaotong University, in 2019. She is now working toward the PhD degree with the School of Intelligent Systems Engineering, Sun Yat-sen University. Her current research interests include intelligent transportation system, multi-agent reinforcement learning and graph neural networks.View more
Temple University, Philadelphia, PA, USA
Krishna Kant (Fellow, IEEE) received the PhD degree in mathematical sciences from the University of Texas at Dallas, in 1981. He is currently a professor with the Computer and Information Science Department, Temple University, Philadelphia, PA, where he directs the IUCRC center on Intelligent Storage. Earlier he was a research professor with the Center for Secure Information Systems, George Mason University. From 2008 to ...Show More
Krishna Kant (Fellow, IEEE) received the PhD degree in mathematical sciences from the University of Texas at Dallas, in 1981. He is currently a professor with the Computer and Information Science Department, Temple University, Philadelphia, PA, where he directs the IUCRC center on Intelligent Storage. Earlier he was a research professor with the Center for Secure Information Systems, George Mason University. From 2008 to ...View more
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Guangzhou, China
Wanquan Liu (Senior Member, IEEE) received the BSc degree in applied mathematics from Qufu Normal University, P. R. China, in 1985, the MSc degree in control theory and operation research from the Chinese Academy of Science, in 1988, and the PhD degree in electrical engineering from Shanghai Jiaotong University, in 1993. He once held the ARC Fellowship, U2000 Fellowship, and JSPS Fellowship and attracted research funds fr...Show More
Wanquan Liu (Senior Member, IEEE) received the BSc degree in applied mathematics from Qufu Normal University, P. R. China, in 1985, the MSc degree in control theory and operation research from the Chinese Academy of Science, in 1988, and the PhD degree in electrical engineering from Shanghai Jiaotong University, in 1993. He once held the ARC Fellowship, U2000 Fellowship, and JSPS Fellowship and attracted research funds fr...View more
Contents School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Guangzhou, China
Linsi Lan received the bachelor's degree in telecommunication engineering from the Minzu University of China, Beijing, China, in 2021. She is currently working toward the master's degree with the School of Intelligent Systems Engineering, Sun Yat-sen University, Guangzhou, China. Her research interests include federated learning, internet of vehicles and autonomous driving.
Linsi Lan received the bachelor's degree in telecommunication engineering from the Minzu University of China, Beijing, China, in 2021. She is currently working toward the master's degree with the School of Intelligent Systems Engineering, Sun Yat-sen University, Guangzhou, China. Her research interests include federated learning, internet of vehicles and autonomous driving.View more
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Guangzhou, China
Junbo Wang (Member, IEEE) received the BS and MS degrees in electrical and electronic engineering from Yanshan University, China, and the PhD degree in computer science and engineering from the University of Aizu, Japan, in 2011. He was a postdoctoral scholar and associate professor with the Univeristy of Aizu, Japan. Currently he is an associate professor with the School of Intelligent Systems Engineering, Sun Yat-sen University, China. He was the PI with Japan site for JST-NSF Joint Funding Program to study Big Data and Disaster (SICORP Project). Currently his research interests include collaborative ML, federated learning, fog computing, Big Data and privacy.
Junbo Wang (Member, IEEE) received the BS and MS degrees in electrical and electronic engineering from Yanshan University, China, and the PhD degree in computer science and engineering from the University of Aizu, Japan, in 2011. He was a postdoctoral scholar and associate professor with the Univeristy of Aizu, Japan. Currently he is an associate professor with the School of Intelligent Systems Engineering, Sun Yat-sen University, China. He was the PI with Japan site for JST-NSF Joint Funding Program to study Big Data and Disaster (SICORP Project). Currently his research interests include collaborative ML, federated learning, fog computing, Big Data and privacy.View more
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Guangzhou, China
Zhi Li received the bachelor's degree from the School of Traffic and Transportation, Beijing Jiaotong University, in 2019. She is now working toward the PhD degree with the School of Intelligent Systems Engineering, Sun Yat-sen University. Her current research interests include intelligent transportation system, multi-agent reinforcement learning and graph neural networks.
Zhi Li received the bachelor's degree from the School of Traffic and Transportation, Beijing Jiaotong University, in 2019. She is now working toward the PhD degree with the School of Intelligent Systems Engineering, Sun Yat-sen University. Her current research interests include intelligent transportation system, multi-agent reinforcement learning and graph neural networks.View more
Temple University, Philadelphia, PA, USA
Krishna Kant (Fellow, IEEE) received the PhD degree in mathematical sciences from the University of Texas at Dallas, in 1981. He is currently a professor with the Computer and Information Science Department, Temple University, Philadelphia, PA, where he directs the IUCRC center on Intelligent Storage. Earlier he was a research professor with the Center for Secure Information Systems, George Mason University. From 2008 to 2013 he served as a program director with NSF where he managed the computer systems research program and was instrumental in the development and running of NSF-wide sustainability initiative named science, engineering and education for sustainability (SEES). Prior to NSF, he served in industry for 18 years (with Intel, Bellcore, and Bell Labs) and 10 years in academia (with Penn State and Northwestern Univ.). He carries a combined 41 years of experience in academia, industry, and government. He has published in a wide variety of areas in computer science, authored a graduate textbook on performance modeling of computer systems. His research interests span a wide range including data center storage and networking, communications in challenging environments, and robustness and security in cyber and cyber-physical systems.
Krishna Kant (Fellow, IEEE) received the PhD degree in mathematical sciences from the University of Texas at Dallas, in 1981. He is currently a professor with the Computer and Information Science Department, Temple University, Philadelphia, PA, where he directs the IUCRC center on Intelligent Storage. Earlier he was a research professor with the Center for Secure Information Systems, George Mason University. From 2008 to 2013 he served as a program director with NSF where he managed the computer systems research program and was instrumental in the development and running of NSF-wide sustainability initiative named science, engineering and education for sustainability (SEES). Prior to NSF, he served in industry for 18 years (with Intel, Bellcore, and Bell Labs) and 10 years in academia (with Penn State and Northwestern Univ.). He carries a combined 41 years of experience in academia, industry, and government. He has published in a wide variety of areas in computer science, authored a graduate textbook on performance modeling of computer systems. His research interests span a wide range including data center storage and networking, communications in challenging environments, and robustness and security in cyber and cyber-physical systems.View more
School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology, Guangzhou, China
Wanquan Liu (Senior Member, IEEE) received the BSc degree in applied mathematics from Qufu Normal University, P. R. China, in 1985, the MSc degree in control theory and operation research from the Chinese Academy of Science, in 1988, and the PhD degree in electrical engineering from Shanghai Jiaotong University, in 1993. He once held the ARC Fellowship, U2000 Fellowship, and JSPS Fellowship and attracted research funds from different resources more than 3.5 million Australian dollars. He is currently a professor with the School of Intelligent Systems Engineering, Sun Yat-sen University, and is the editor-in-chief for the International Journal Mathematical Foundation of Computing and on the editorial board for several international journals. His current research interests include intelligent control systems, pattern recognition, machine learning, and computer vision.
Wanquan Liu (Senior Member, IEEE) received the BSc degree in applied mathematics from Qufu Normal University, P. R. China, in 1985, the MSc degree in control theory and operation research from the Chinese Academy of Science, in 1988, and the PhD degree in electrical engineering from Shanghai Jiaotong University, in 1993. He once held the ARC Fellowship, U2000 Fellowship, and JSPS Fellowship and attracted research funds from different resources more than 3.5 million Australian dollars. He is currently a professor with the School of Intelligent Systems Engineering, Sun Yat-sen University, and is the editor-in-chief for the International Journal Mathematical Foundation of Computing and on the editorial board for several international journals. His current research interests include intelligent control systems, pattern recognition, machine learning, and computer vision.View more
