A Data Reconstruction Attack Against Vertical Federated Learning Based on Knowledge Transfer

Vertical federated learning allows multiple parties, each possessing data on different features of the same samples, to collaboratively train a machine learning model while keeping their data private. Naive vertical federated learning systems, however, face severe limitations degrading their usability; for example, they require all parties to be involved even for basic inference tasks. A state-of-the-art approach overcomes the limitations by incorporating knowledge distillation techniques based on federated singular value decomposition. This allows parties to collaboratively derive latent representations of their data without exposing the actual data to each other. Each party can then independently train a model and perform inference on that model, enhancing the practicality of vertical federated learning. Although the original data of each party is not exposed to other parties during these procedures, we reveal that a semi-honest adversary can potentially reconstruct the original data from its latent representation. The proposed attack exploits the linear relationship between the original data and the corresponding latent representation and deduces the relationship using a metaheuristic approach based on a GAN-like neural network. The effectiveness of the attack is validated using several well-known real-world datasets.