Gamekeeper: Online Learning for Admission Control of Networked Open Multiagent Systems

We consider open games where players arrive according to a Poisson process with rate
$\lambda$
and stay in the game for an exponential random duration with rate
$\mu$
. The game evolves in continuous time where each player
$n$
sets an exponential random clock and updates her action
$a_{n}\in \lbrace 0,\ldots,K\rbrace $
when it expires. The players take independent actions according to local decision rules that, uninterrupted, are designed to converge to an equilibrium. This models open multiagent systems such as in wireless networks, transportation, cloud computing, and online marketplaces. When
$\lambda$
is small, the game spends most of the time in a (time-varying) equilibrium. This equilibrium exhibits predictable behavior and can have performance guarantees by design. However, when
$\lambda$
is too small, the system is under-utilized since not many players are in the game on average. Choosing the maximal
$\lambda$
that the game can support while still spending a target fraction
$0< \rho < 1$
of the time at equilibrium requires knowing the reward functions. To overcome that, we propose an online learning algorithm that the gamekeeper uses to adjust the probability
$\theta$
to admit an incoming player. The gamekeeper only observes whether an action was changed, without observing the action or who played it. We prove that our algorithm learns, with probability 1, a
$\theta ^{*}$
such that the game is at equilibrium for at least
$\rho$
fraction of the time, and no more than
$\rho +\varepsilon (\mu,\rho)< 1$
, where we provide an analytic expression for
$\varepsilon (\mu,\rho)$
. Our algorithm is a black-box method to transfer performance guarantees of distributed protocols from closed systems to open systems.