Towards Fair and Scalable Trial Assignment in Federated Bandits: A Shapley Value Approach

Federated multi-armed bandits extend the multi-armed bandits framework to the federated learning setting where multiple clients in the same exploration space collaboratively identify the optimal arm. While previous studies demonstrated its efficiency like classical federated learning, in this paper, we present the first work that reveals the serious fairness problem in federated multi-armed bandits when clients have heterogeneous and overlapping armsets. The fairness problem happens because clients with different trial requirements should conduct different numbers of trials summing up to a global requirement, but they wish to minimize their exploration effort. To address the novel fairness concern, we formally formulate the fairness-aware trial assignment as a coalitional game. Based on the theoretically derived trial requirements, we devise a Shapley value-based trial assignment mechanism to guarantee fairness. Regardless of the #P-hard complexity when deriving the general Shapley value, we achieve an accurate computation of trial assignment with polynomial complexity by exploiting its unique characteristic. We further carefully control the numbers of trials in each iteration to resolve the communication bottleneck and minimize the wasted trials. Experiment results show that, compared to the naïve federated scheme, our design outperforms with both high fairness metrics and high efficiency in total trials and communication.