A Sum-of-States Preservation Framework for Open Multiagent Systems With Nonlinear Heterogeneous Coupling

In this article, we develop a general open multiagent systems (OMAS) framework over undirected graphs where the agents' interaction is, in general, nonlinear, time-varying, and heterogeneous, in that the agents interact with different pairwise interaction rules for each link, possibly nonlinear, which may change over time. In particular, assuming the agents interact by exchanging flows, which modify their states, our framework guarantees that the sum of the states of agents participating in the network is preserved. To this end, agents maintain a state variable for each of their neighbors. Upon the disconnection of a neighbor, such a variable is used to completely eliminate the effect of previous interaction with disconnected agents from the overall system. In order to demonstrate the effectiveness of the proposed OMAS framework, we provide a case study focused on average consensus, and, specifically, we develop a sufficient condition on the structure of the agents' interaction guaranteeing asymptotic convergence under the assumption that the network becomes fixed. The article is complemented by simulation results that numerically demonstrate the effectiveness of the proposed method.


I. INTRODUCTION
Open multiagent systems (OMAS) represent a generalization of multiagent systems (MAS) where agents may join or leave the network.Sensor networks, in which nodes' batteries may run out (e.g., see [1], where a protocol aimed at maximizing the lifetime of a wireless sensor network is presented), and mobile robot networks, in which agents could be temporarily collaborating to achieve an objective during their exploration of an environment (e.g., see [2], where mobile robots form temporary chains of agents to find a path), are examples of such systems.Other interesting examples include precision farming applications, where autonomous intelligent drones, which are capable of actively monitoring a field in order to identify and map features of interest (e.g., weeds or pests) that could be distributed heterogeneously within the field, may join and leave the network over time due to their limited battery autonomy or vehicular ad hoc networks (VANETs), where cars may participate in the network only for a limited amount of time (e.g., see [3], where a clustering and cluster head selection algorithm is provided for open intervehicular networks).
In this article, we propose an OMAS framework for undirected networks, where the pairwise interaction among agents is modeled by a nonlinear function that may change over time.Our approach, in particular, ensures that the sum of the states of the agents currently participating in the network is preserved.In this view, the cornerstone of the proposed framework is represented by state augmentation, in which agents maintain an additional "storage" state variable for each of their neighbors.Such a variable is used to cancel out the effect of the previous interactions with neighboring agents that left the network.Notice that, to the best of our knowledge, this is the only work addressing consensus for open multiagent systems with nonlinear and time-varying coupling.The nonlinear, time-varying, and heterogeneous nature of the interaction rules considered in this brief article have the potential to yield better performance, for instance in terms of error rejection, while allowing agents to join and leave the network at will.In order to show the effectiveness of the proposed framework, we provide a case study in the context of the well-known average consensus problem and, under the premise that the network becomes fixed, we develop a sufficient condition that assures asymptotic convergence.

A. State of the Art
In the literature, several works on MAS have focused on the possibility that agents may join or leave the network.For example, in [4], the problem of adaptive coalition formation is considered; in [5], the authors develop a trust and reputation model for OMAS; [6] presents an OMAS gossiping framework; the works in [7] and [8] focus on the ability of agents in an OMAS setting to form short-term teams; in [9], the stability of gradient descent for OMAS is discussed.Notably, consensus and, in particular, average consensus, represents a popular topic in the context of OMAS, and several distributed consensus algorithms that explicitly account for agents that may join or leave the network have been proposed in the literature.In particular, [10] provides a dynamic average consensus algorithm that is robust to the dynamic change of communication topologies as well as the joining and leaving of nodes; however, the algorithm guarantees convergence up to a nonzero steady-state bounded error.In [11] and [12], a plug-and-play distributed architecture for model predictive control and distributed Kalman filtering is presented, respectively.In [13], the authors develop an algorithm based on the premise that agents leave and arrive at predetermined periods, whereas [14] assumes that each time an agent departs the network, another one enters it instantly.The technique described above has been extended to the case of time-varying network size in [15].The case where agents need to estimate the time-varying average of a set of reference signals is addressed in [16] and [17].In [18], stochastic consensus for OMAS is investigated under the assumption that arrivals and departures occur randomly as a Bernoulli process.In [19], the authors propose an OMAS consensus process in which agents track the median of time-varying reference signals.Agent interactions over randomly induced discretized Laplacians are investigated in [20].In [21], multidimensional switched systems are used to characterize an OMAS.Under the assumption of frequent arrivals and departures of agents, the work in [22] characterizes the performance limitations of average consensus in an OMAS setting, establishing lower bounds on the predicted mean squared error.Moreover, in [23] an OMAS strategy to compute the mode of the agents' state is proposed.This approach is based on a novel OMAS average consensus algorithm, which, under the assumption that the overall number of agents is fixed, guarantees that the effect of agents leaving the network is ruled out.
Finally, it is worth mentioning that, although not intended for OMAS scenarios, in the literature, some average-preserving protocol approaches have been developed, based on auxiliary variables, sometimes also referred to as "storage variables" and "surplus variables" [24], [25].However, so far, only linear state update strategies for these auxiliary/storage variables have been considered.

B. Contribution
In this article, we develop an OMAS framework where the pairwise agents' interaction is in general nonlinear, time-varying, and heterogeneous.To this end, we present the agents' interaction in terms of flows and divergence. 1 Then, in order to show the potential of the approach, we consider the average consensus problem as a valuable case study, and we show that such a framework preserves the sum of the values chosen by the agents at the last instant they join the network.The proposed framework relies on support variables that accumulate the flows received by neighboring agents.Interestingly, this accumulation is possible in spite of the nonlinearity and time-variability of the exchanged flows.In more detail, similarly to the approach in [23] for the linear case, in this article we assume that each agent maintains an additional state variable for each of its neighbors and that, upon disconnection of a neighbor, such a variable is used to rule out the influence of the disconnected neighbor.Notably, the proposed framework extends [23] in a number of ways; in particular, we allow for nonlinear, time-varying, and heterogeneous interaction schemes while no assumption is made on the number of agents.

C. Article Outline
The rest of this article is organized as follows.In Section II, we provide a detailed discussion of the objective of the article.In Section III, we describe how the proposed framework may represent a generalization of the current studies on MAS.In Section IV, we characterize the ability of our framework to preserve the sum of the states of the agents currently participating in the network.Section V considers the average consensus problem as a case study and develops a sufficient condition to guarantee asymptotic convergence when the topology becomes fixed.Section VI provides a simulation campaign 1 The divergence is an operator that provides a measure of the rate of variation of a quantity defined on a node in a network (e.g., see [26]).When applied to the agents' states it essentially corresponds to the difference between the outgoing flow from the node to its neighbors and the incoming flow to the node from its neighbors.
aimed at numerically demonstrating the effectiveness of the proposed approach.Finally, Section VII concludes this article.

II. OMAS: A GENERALIZATION OF MAS
Let us consider a nominal MAS system where agents interact over a fixed graph G = {V, E} with n nodes V = {v 1 , v 2 , . . ., v n } and e edges E ⊆ V × V , where (v i , v j ) ∈ E captures the existence of a link from node v i to node v j .Moreover, let us assume G is undirected, i.e., (v i , v j ) ∈ E whenever (v j , v i ) ∈ E and connected, i.e., each node v i can be reached from each other node v j using the edges in E. Let a ij be such that a ij = 1 if (v i , v j ) ∈ E and a ij = 0, otherwise.Moreover, let f k ij denote the, possibly, nonlinear and time-varying flow from i to j at the kth step, e.g., a value or quantity sent from agent i to agent j at step k.In particular, let us define the in-flow and out-flow for agent i at step k as respectively.In other words, the above quantities correspond, respectively, to the sum of the incoming or outgoing flows at step k.Moreover, the divergence associated to agent i at step k is defined as the imbalance between the out-and in-flow at step k, i.e., Finally, let us define the total divergence DIV k at step k, i.e., the total variation of the system due to the flows, as Notably, since the underlying graph G is undirected, by construction we have that, for any choice of the terms f k ij , it holds DIV k = 0. Based on the above definitions, let us consider agents interacting over G according to the following discrete-time dynamics: Notice that, in spite of the generality of the dynamics and, in particular, of the flows, we have that , the sum of the states is preserved, and thus Moreover, let us assume that the above dynamics converges to a function χ(•) of the initial states of all agents, i.e., lim t→∞ Let us now discuss how the above dynamics behave in an OMAS setting.Notably, in this case, the agents exchange the flows f k ij only with neighbors that are currently participating in the network.In this case, the preservation of the sum of the states and the convergence to χ(•) is no longer granted due to the variation in the set of agents occurring over time.
In this view, the aim of this article is to develop a framework to extend the peculiarities of the above MAS dynamics to an OMAS setting, guaranteeing that the sum of the states of the agents currently participating is preserved and that, in the event that the topology becomes fixed, the agents are able to compute χ(•) over the initial states of the agents currently participating in the network, as it would Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
occur within a typical MAS setting.In other words, our objective is to make sure that the effect of agents joining the network is taken into account, while the effect of agents leaving the network is completely ruled out.

III. PROPOSED OMAS FRAMEWORK
Let us consider a scenario where a network of agents interacts in a synchronous discrete-time fashion in an OMAS setting.In particular, we assume that each agent can join and/or leave multiple times.In this view, each agent i is assumed to join or leave at given steps and, in particular, is characterized by the sets A i , D i ⊂ N ≥0 , i.e., the sets collecting the steps at which agent i joins and leaves (i.e., ∀i, h τ A,h i < τ D,h i ), respectively.Notably, the ith agent is active at those steps k such that Let us assume that when an agent joins the network, it creates undirected links arbitrarily and when it leaves, all its links are removed.Therefore, in the considered setting, the agents interact according to a time-varying graph and, specifically, we use G k = {V k , E k } to denote the graph underlying the agents' interaction at step k.Notice that G k is assumed to be undirected but it can be disconnected.
Briefly, in this article, we assume that, when an agent joins the network at some step τ , it joins with an arbitrary value; in the following, we use x τ i to denote the value chosen by the i-th agent when it (re)activates at step τ .Notice that, where understood, we simply use x i to denote the value chosen at the last (re)activation step.Let us now define a few variables that will be used as index functions to denote the agents arriving, departing, or remaining in the network, respectively, i.e., In other words, α k i , ζ k i , and θ k i are equal to one if the ith agent is joining, leaving, or active at time k, respectively, and are zero otherwise.Moreover, let us use a k ij to denote the existence of a link between i and j at step k.Notably, based on the above variables, a k+1 ij can be expressed as follows: Clearly, since the agents create undirected links, we have that a k ij = a k ji .At this point, considering the flows f k ij , the in-flow and out-flow for agent i at step k can be rewritten as respectively.Moreover, the divergence at agent i at step k is rewritten as Finally, the total divergence DIV k at step k, is modified accordingly, i.e., Notably, also in this case, since the underlying graph G k is undirected, by construction, for any choice of the terms f k ij , it holds DIV k = 0. Based on the above definitions, we now develop a strategy to extend the nominal MAS dynamics in (1) to an OMAS setting, which will be proven to preserve the sum of the values chosen by each of the agents currently participating in the network at its last (re)activation step, in spite of activations and deactivations.In particular, we consider the following dynamics for the agents: with x 0 i = x 0 i and z 0 ji = 0. Briefly, the term α k i x i accounts for case where the ith agent activates at step k and selects a value x i .Moreover, the term i models the nominal dynamics involving the agent and its neighbors, when the agent is already present in the network at step k.Finally, the terms −(1 − a k+1 ij )z k ij account for the sum of the flows exchanged by a neighbor j that leaves the network at step k + 1, and is introduced in order to get rid of the cumulative/past contribution of disconnecting agents at later times.In more detail, the terms z k ji represent additional state variables that each agent maintains for each of its neighbors.Notice that, according to (2), when an agent joins the network at step k it sets z k+1 ij = 0. Notice further that deactivated agents may either stop updating their values or set them to some arbitrarily chosen values.

IV. SUM-OF-STATES PRESERVATION
The MAS dynamics considered in this article are very general, since the structure of the terms f k ij is intentionally not further specified.In spite of its generality, we now show that the extension to an OMAS setting given in ( 2) is such that the sum of the states of agents currently participating in the network equals the sum of the values they chose at the last step in which they joined the network.
Theorem 1: Let us consider an OMAS where agents interact according to the dynamics given in (2).Moreover, let us assume that each agent, upon joining the network at a time step τ , selects an arbitrary value x τ i that represents its initial condition (or a reinitialized initial condition) of the value of the agent joining, and let x i denote the value chosen at the last time instant at which the agent i joins the network, i.e., step τ that is closest to the current time instant k.At each step, k the sum of the states of the active agents is equal to the sum of the values x i , i.e., i Proof: In order to prove our statement we observe that at k = 0, by construction it holds hence, the statement holds true at k = 0. Let us now prove the statement holds at k + 1 for all k ≥ 0. Notice that, by using the dynamics for z k ij Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
in (2) we have that where we used the fact that, by construction, it holds (a k ij ) 2 = a k ij .Therefore, for all i such that (1 − α k i )θ k i = 1, by using the dynamics for x k i in (2) and the above equation, it holds Thus, considering the largest step k * i ∈ A i with k * i ≤ t (which always exists by construction), we have that where the latter equality holds since, by construction, all terms z Moreover, by construction, for all i such that α k i = 1 it holds x k+1 i = x i and z k+1 ij = 0. Therefore, noting that the agents with θ k+1 i = 1 are either those such that α k i = 1 or those such that The proof follows noting that, by definition, z k+1 ij = 0 whenever θ k+1 i = 0, and thus where the latter equality holds since, by construction, z k ij = −z k ji and the graph G k is undirected for all k.The proof is complete.
We established that, by resorting to the proposed framework, the sum of the states of a generic distributed system based on the exchange of flows among the agents is preserved in spite of the openness of the system.We reiterate that the agents' dynamics are very general and, in particular, the structure of the flows is intentionally not further specified.Therefore, our framework represents a viable way to extend the dynamics originally developed for an MAS context as in (1), in order to account for the possibility that agents may join or leave the network during the evolution.

V. CASE STUDY: AVERAGE CONSENSUS
Notice that the proposed OMAS strategy applies to a broad variety of situations, preserving the sum of the agents' initial state.In this section, we focus on average consensus as a representative problem instantiation of the proposed framework and we inspect the case of both linear and nonlinear flows.In particular, in order to characterize a class of systems that reaches the average of the initial conditions when the topology becomes fixed, let us consider the following assumption.
Assumption 1: There is a finite step k † such that it holds G k = G k † for all k ≥ k † , i.e., no activation or deactivation occurs from step k † on.
In the following, we assume G k † is composed of m connected components and we use V h to denote the set of agents in the hth component, while we use ψ i to denote the identifier of the connected component featuring the ith agent.
In order to develop a sufficient condition that, when the network stops changing, guarantees the reach of the average of the initial values of the agents participating in the network, let us now introduce a further assumption on the structure of the flows f k ij .Assumption 2: For all steps k ≥ 0 and for each unordered pair of nodes {v i , v j } such that (v i , v j ), (v j , v i ) ∈ E k , the flows satisfy where: 1) the functions g k {i,j} (x k i − x k j ) are odd, i.e., 2) g k {i,j} (•) is zero only at zero; 3) for where | • | is the absolute value and Notably, we assume that at each time step each link has, in general, different interaction rules g k {i,j} (•), even though the interaction is skewsymmetric at the level of each link.
Notice that points (1) and ( 2) are classical requirements in the context of MAS (e.g., [27] and [28]); in particular, point (1) comes from the requirement that the interaction is symmetrical and point ( 2) is due to the desire that the agents stop interacting when they reach the same value.Regarding the last requirement, since the function is zero at zero, this requirement is satisfied when the functions g k {i,j} (•) are Lipschitz.Interestingly, the class of flows that satisfy Assumption 2 is quite large and features, for instance, the functions reported in ( 6)-( 9), which include the classical linear interaction [see (6)] as well as functions that account for saturations [either smooth as in (7) and (9) or nonsmooth as in (8)].
We now establish that under Assumptions 1 and 2, the state of each agent converges to the average of the initial conditions of the set of Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
agents belonging to the same connected component.To this end, we first need the following ancillary lemma.
Lemma 1: Let us consider an OMAS system where agents interact according to the dynamics given in (2) and let Assumptions 1 and 2 hold true.For all steps k ≥ k † it holds Proof: In order to prove the statement we observe that, for k ≥ k † the graph G k is fixed and is equal to and let Γ k i denote the n k × n k matrix with the ith row that coincides with the ith row of Ω k , while all other entries are equal to zero.We have that where the last equality holds since G k is undirected and g k {i,j} (•) is odd.This completes our proof.
We are now in a position to prove convergence when the agents' topology becomes fixed.
Theorem 2: Let us consider an OMAS where agents interact according to the dynamics given in (2) and let Assumptions 1 and 2 hold true.Then, all agents i for which θ k † i = 1 converge to the average of the initial values x j of the agents in the set V ψ i , i.e., lim t→∞ where card(•) denotes the cardinality of a set.
Proof: For the sake of simplicity and without loss of generality, let us consider the case where there is only one connected component (otherwise, the reasoning of this proof can be applied to each connected component).In this case, for all i ∈ V k † , it holds In order to prove convergence of the agents' states to x, let us consider the Lyapunov-like function where x is a generic value to be determined later in the proof.Notice that the above function is zero only when all x k i = x and is positive otherwise.At this point we observe that, by construction, it holds Moreover, by construction, div k i = 0 when θ k † i = 0; therefore, we have that where the latter equality holds since the total divergence DIV k is zero.As a consequence, since by Assumption 1 the graph is fixed for all k ≥ k † , we have that Let us now define ΔW k = W k+1 − W k .We have that where we used the fact that, for all k, G k is undirected and g {i,j} (•) is odd.At this point, we observe that by construction div k i = 0 when Hence, by using Lemma 1, we have that Since, by Assumption 2, the terms g k (•) are odd functions and are zero only at zero, we have that, unless Therefore, we have that, for x k i = x k j , the terms ΔW k ij are negative iff (3) holds true.Since, by Assumption 2, this is the case, we conclude that, unless all x k i such that v i ∈ V k † are equal to x, the term ΔW k is negative.We have established that the state of all agents that participate in the network at step k † converges to the same value x.To conclude the proof, let us now show that it must hold x = x.To this end, we observe that since in Theorem 1 we established that the sum of the initial conditions is preserved, by definition it must hold This completes our proof.
The next remark characterizes a broad class of systems that satisfy Assumption 2, and thus converge to the average when the topology becomes fixed.
Remark 3: Assumption 2 holds true when the odd functions g k {i,j} (•) are locally Lipschitz with Lipschitz constant k {i,j} < 1/δ k {i,j} .In fact, since g k {i,j} (•) is zero at zero, we have that Examples of flows belonging to this class include, among other possibilities, the following cases: In all the above cases, it can be shown that k {i,j} = w k {i,j} and thus, the assumption holds by choosing weights 0 < w k {i,j} < 1/δ k {i,j} .

VI. SIMULATIONS
In order to numerically demonstrate the effectiveness of the proposed approach, we consider a case where G 0 is an Erdös-Renyí graph with n = 30 nodes and link formation probability p = 0.3 (not reported for space reasons).Moreover, we assume that at step k = 59, a subset of five agents becomes disconnected, and is reconnected at step k = 99, while another set of ten agents is disconnected at step k = 69 and reconnected at step k = 149.Notably, at all steps k, the agents that are active always belong to the same connected component.We assume the agents' initial condition x 0 i is chosen uniformly at random in [0, 100] and, in particular, we have that the average of the initial conditions is i x 0 i = 51.2583.Moreover, we assume that, when an agent is reconnected at step k ‡ , it selects again the original initial condition, i.e., x k ‡ i = x 0 i .Finally, we assume that for each step k and for each link (v i , v j ) ∈ E k , the pairwise interaction rule g k {i,j} (•) is selected at random from those in (6)- (9).In particular, we set w k {i,j} = 1/(1 + δ k {i,j} ) when g k {i,j} (•) is selected as in ( 6)-( 9).Fig. 1 reports the evolution of the MAS dynamics in (1), when the terms g k {i,j} (•) are chosen as above (for the sake of readability, disconnected agents maintain their last updated states when disconnected).Conversely, Fig. 2 shows the evolution of the agents' states in the proposed OMAS setting.It can be noted that, while the MAS dynamics fail to track the average of their initial states (shown by gray asterisks),  the proposed OMAS framework is successful in accomplishing the task.   in the OMAS at their last joining instant is preserved up to numerical precision, thus experimentally validating Theorem 1; conversely, the MAS dynamics do not preserve the sum of the agents currently participating in the network.Finally, Fig. 4 shows the temporal evolution of the Lyapunov function W k , again, considering both the MAS and OMAS settings.Notably, when at the beginning no agent joins/leaves, the evolution of W k is the same for both the MAS and OMAS dynamics.However, the addition/removal of agents generates new transients: while in the MAS setting the Lyapunov function fails to converge to zero, in the OMAS setting we observe that, after each transient, the states of the active agents approach the average of the states currently participating in the network.

VII. CONCLUSION
This article presents an OMAS framework for undirected networks with nonlinear and time-varying agent interactions.Our method, in particular, ensures that the sum of the present states of the agents in the network is preserved.Furthermore, we develop a sufficient condition that ensures asymptotic convergence under the assumption that the network becomes fixed.Future work will aim to extend the proposed framework to directed graphs and to exploit the nonlinear, time-varying, and heterogeneous nature of the interaction rules to improve performance, e.g., in terms of convergence speed, error rejection, resistance to outliers, or distributed stopping.Moreover, we will investigate the possibility of applying this approach to distributed optimization problems.

Fig. 1 .
Fig.1.Example of MAS dynamics when g k {i,j} (•) is randomly selected from the functions in (6)-(9) and agents join and leave the network.

Fig. 2 .
Fig. 2. Example of OMAS dynamics when g k{i,j} (•) is randomly selected from the functions in (6)-(9) and agents join and leave the network.

Fig. 3
reports the temporal evolution of i a k ij x k i − i a k ij x k i both in the MAS and OMAS examples.According to the figure, in spite of the variability of the network and of the different choices for g k {i,j} (•), the sum of the states chosen by agents currently participating Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

Fig. 3 .
Fig.3.In the OMAS setting, the sum of the initial states of the active agents is preserved, up to numerical precision; the MAS dynamics fail to do so.

Fig. 4 .
Fig. 4. Evolution of the Lyapunov function W k , in both a MAS and OMAS setting.
using • 2 and • F to denote the Euclidean and Frobenius norms, respectively, it holds