Dynamics of a Stratified Population of Optimum Seeking Agents on a Network—Part I: Modeling and Convergence Analysis

In this article, we consider a population composed of a continuum of agents that seek to maximize a payoff function by moving on a network. The nodes in the network may represent physical locations or abstract choices. The population is stratified, i.e., agents opting for the same choice may not get the same payoff. In particular, we assume payoff functions with diminishing returns, i.e., agents in “newer” strata of a node receive a smaller payoff compared to “older” strata. In this first part of this two-part work, we model the population dynamics under three choice revision policies, having varying levels of coordination: 1) no coordination and the agents are selfish; 2) coordination among agents in each node; and 3) coordination across the entire population. For the case with selfish agents, we generalize the Smith dynamics to our setting, where we have a stratified population and network constraints. To model nodal coordination, we allow the fraction of population in a node, as a whole, to take the “best response” to the state of the population in the node’s neighborhood. For the case of population-wide coordination, we explore a dynamics where the population evolves according to centralized gradient ascent of the social utility, though constrained by the network. In each case, we show that the dynamics has existence and uniqueness of solutions and also show that the solutions from any initial condition asymptotically converge to the set of Nash equilibria.