Decentralized Multi-Agent Bandit Learning for Intelligent Internet of Things Systems

In intelligent Internet of Things systems, data-hungry services are empowered by data collection, which is jointly accomplished by edge servers and data-collecting sensors. In this paper, we aim to achieve efficient data collection, i.e., maximize data rates from sensors to servers while mitigating the impact of data heterogeneity for data collected from sensors. Considering geographically distributed servers and sensors, we study the problem from the perspective of multi-agent multi-armed bandits. The key ideas of our approach are to 1) establish associations between servers and sensors under unknown wireless dynamics (i.e., channel state information) and selection fraction constraints; 2) utilize shared information via pairwise communication between servers to mitigate biased observations for data rates. To this end, we propose a scheme that leverages online learning to reduce uncertainties in wireless dynamics and online control to mitigate the impact of data heterogeneity. Based on an effective integration of bandit learning methods under pairwise communication and Lyapunov optimization techniques, we present a novel Decentralized sErver-Sensor association scheme with Multi-Agent learning under pairwise communication (DESMA). Our theoretical analysis demonstrates that DESMA achieves a tunable trade-off between maximizing data rate and mitigating the impact of data heterogeneity.