Heterogeneous Defect Prediction Based on Federated Reinforcement Learning via Gradient Clustering

Heterogeneous defect prediction (HDP) refers to using heterogeneous data collected by other projects to build a defect prediction model to predict the software defects in a project. Traditional methods usually involve the measurement of the source project and the target project. However, due to the limitations of laws and regulations, these original data are not easy to obtain, which forms a data island. As a new machine learning paradigm, federated learning (FL) has great advantages in training heterogeneous data and data island. In order to solve the data island and data heterogeneity of HDP, we propose a novel Federated Reinforcement Learning via Gradient Clustering (FRLGC) method in this paper. Firstly, the parameters of the global model are transferred to each dueling deep Q network (dueling DQN) model and each client uses private data to train the dueling model which combines experience replay to increase data efficiency in limited datasets. Secondly, gaussian differential privacy is used to encrypt the model parameters to ensure the privacy and security of the model. Finally, we cluster the clients according to their locally encrypted model parameters and use weighted average to aggregate to create a new global model locally and globally. Experiments on nine projects in three public databases (Relink, NASA and AEEEM) show that FRLGC is superior to the relevant HDP solutions.