FedTLBOHB: Efficient Hyperband with Transfer Learning for Vertical Federated Learning

The issue of data privacy is receiving increasing attention nowadays. Federated learning, which is a distributed machine learning setting where many clients (e.g. mobile devices or organizations) train a model collaboratively while keeping the training data local, was born as a result. And it has been applied to an increasing number of business scenarios. Like traditional machine learning, federated learning methods are also very sensitive to hyperparameters. However, data privacy will be protected using cryptographic techniques such as homomorphic encryption and secret sharing in federated learning, resulting in significant data amounts transferred among clients. Given this, we should find a more efficient method to optimize hyperparameters, which means utilizing fewer budgets (time, iterations, etc.). In this paper, we proposed a new efficient method FedTLBOHB, which combines both Hyperband and knowledge transfer for model-based optimization, to optimize hyperparameters of vertical federated learning. FedTLBOHB achieve high security level by avoiding a meta-feature based transfer learning paradigm, which will bring about security concerns in federated learning. Instead, a method similar to the model ensemble in Adaboost is designed for knowledge transfer. Meanwhile, according to our analysis, FedTLBOHB outperforms BOHB. The proposed method has been tested extensively on OpenML datasets and has been shown to be more effective than conventional methods at finding better hyperparameter configurations.