An Integrated Human–Cyber–Physical Framework for Control of Microgrids

In this paper, to jointly study the energy dynamic behavior of humans and the corresponding physical dynamics of the microgrid, we bridge two disciplines: systems & control and environmental psychology. Firstly, we develop second order motivation-behavior mathematical models inspired by opinion dynamics models for describing and predicting human activities related to the use of energy, where psychological variables and social interactions are considered. Secondly, based on these models, we develop a human-cyber-physical system framework consisting of three layers: (i) human, (ii) cyber and (iii) physical. The first one describes human behavior influenced by behavioral intervention and motivation, which in turn depend on contextual factors, personal values and social norms. The cyber layer solves an optimization problem and embeds load controllers, which are designed to automatically mimic human behavior. Finally, the physical layer represents an AC microgrid. Thus, we formulate a social-physical welfare optimization problem and solve it by designing a distributed primal-dual control scheme, which generates the optimal behavioral intervention (with respect to a given reference) and the control inputs to the microgrid.

Abstract-In this paper, to jointly study the energy dynamic behavior of humans and the corresponding physical dynamics of the microgrid, we bridge two disciplines: systems & control and environmental psychology. Firstly, we develop second order motivation-behavior mathematical models inspired by opinion dynamics models for describing and predicting human activities related to the use of energy, where psychological variables and social interactions are considered. Secondly, based on these models, we develop a human-cyber-physical system framework consisting of three layers: (i) human, (ii) cyber and (iii) physical. The first one describes human behavior influenced by behavioral intervention and motivation, which in turn depend on contextual factors, personal values and social norms. The cyber layer solves an optimization problem and embeds load controllers, which are designed to automatically mimic human behavior. Finally, the physical layer represents an AC microgrid. Thus, we formulate a social-physical welfare optimization problem and solve it by designing a distributed primal-dual control scheme, which generates the optimal behavioral intervention (with respect to a given reference) and the control inputs to the microgrid.
Index Terms-Control and psychology, control of microgrids, optimization.

I. INTRODUCTION
I NDIVIDUALS' energy behavior can critically affect the functioning of energy systems. A good understanding of individuals' behavior and its drivers is needed to accurately model and optimize energy systems. Knowledge on such drivers in psychology could be employed to better understand individuals' energy behavior that affects the energy system's functioning, and to promote the behavior that makes the energy system function more optimal, possibly enhancing the effectiveness of technical solutions [1], [2].
In control and optimization problems for microgrids, modeling human behavior has attracted increasing attention. For instance, in [3], based on hidden Markov modeling techniques, the authors have built models capturing variations in consumer habits of daily life. When the consumer requests to activate a time-shiftable appliance, based on the model and control algorithm, the activation request will be accepted or postponed. The work in [4] proposes a model describing the load behavior of individual households using Markov chains, and then develops a control strategy for consumption reduction. In [5], the authors consider Demand Side Management (DSM) to optimize the energy consumption and, based on population dynamics, model the dynamic behavior of humans participating/quitting DSM programs. In [6], an incentive (price) based method is proposed to reduce the peaks of demand. In particular, through smart meters, the users can receive different pricing schemes for optimally managing flexibility.
To simultaneously analyse the energy dynamic behavior of humans and the physical dynamics of the microgrid, we present an interdisciplinary work that integrates systems & control and psychology. We first develop feasible mathematical models describing human activities related to the use of energy. Then, to model the impact of dynamical human behavior on a microgrid and the effects of behavioral interventions (e.g., financial incentives) on energy use, we develop a human-cyber-physical system (HCPS) framework, based on which we obtain the optimal inputs to the microgrid (e.g., to control inverters and controllable loads) as well as suitable behavioral interventions. Specifically, the HCPS includes three layers. The first one is the human layer, which describes human activities; the second one is the cyber layer and consists of an optimal control system; the third one is the physical layer, i.e., an islanded AC microgrid (see Figure 1).
Energy behavior in psychology: The dynamic human behavior models proposed in this paper are inspired by findings in psychology. It is well established that energy use behavior is rooted in an individual's personal values (values for short), which reflect general goals that people strive for in life [2]. Personal values typically influence an individual's energy behavior via motivation, indexing a person's willingness to perform a behavior [7]. This paper considers three types of values that appear particularly relevant to energy behavior: egoistic values that imply  people aim to enhance their resources such as possessions, money and status, hedonic values that imply people aim to increase pleasure and comfort, and biospheric values that reflect a concern to protect nature and the environment [1], [8]. Stronger endorsement of biospheric values likely results in stronger motivation to save energy, which in turn increase the likelihood that someone will actually engage in energy saving behavior. Conversely, stronger endorsement of hedonic values is typically indicative of a weaker motivation to save energy because saving energy may be considered inconvenient or uncomfortable to do, which may demotivate energy saving behavior [9]. Individuals may also be motivated to increase/decrease their energy consumption because of associated behavioral intervention such as subsidies and taxes, which may particularly appeal to people who care relatively much about finances. Moreover, social influence can affect people's energy consumption. For instance, if an individual believes that most others regard saving energy the right thing to do (i.e., injunctive social norm) or see most others to engage in energy saving behaviour (i.e., descriptive social norm), this can motivate them to save energy as well [10], [11]. Also, we briefly discuss the effects of contextual factors on people's motivation and the resulting energy behavior [12].
HCPS framework: Energy behavior is influenced by the underlying motivation and behavioral intervention. From a control perspective, the behavioral intervention can be considered as a "control input" to the individual's behavior dynamics, while motivation can be considered as "reference signal" for the behavior in absence of behavioral intervention. On the other hand, from the viewpoint of opinion dynamics, motivation can be considered as the "opinion" concerning energy behavior of users which may affect others in a social network. For this reason, our models describing motivation are partially inspired by and consistent with studies in opinion dynamics [13], [14].
For the physical layer, we consider an islanded AC microgrid where people (prosumers) generate power within their local electricity facilities (e.g., PV panels, energy storage devices), which are connected through a distribution power network. The prosumers' loads are equipped with local control units that are designed to automatically mimic human behavior influenced by behavioral intervention. The local control units can also exchange necessary information, e.g., current generation, with their neighboring control units in the microgrid and are responsible for regulating the microgrid voltage towards a given value. Thus, the cyber layer (control system and communication network) acts as interface between the human and physical layers, guaranteeing the grid stability and adjusting automatically the loads by taking into account individuals' behavior.
We formulate a convex social-physical welfare optimization problem, whose solution corresponds to "control inputs" to the microgrid (i.e., voltage sources) and load control units (i.e., intervention). We aim at i) maximizing the social welfare by satisfying the prosumers' load demand and minimizing the cost associated to the behavioral intervention; ii) maximizing the physical welfare by guaranteeing grid stability and voltage regulation while minimizing the cost associated to current generation. To achieve these goals, we design a dynamic controller, whose unforced dynamics represent the primal-dual dynamics of the considered optimization problem [15], [16], [17]. The main contributions of this paper are: • We bridge systems & control and psychology by developing a novel HCPS framework to study and analyze dynamic energy behavior of humans, taking into account the dynamics of the physical infrastructures of an islanded AC microgrid. • To connect control and psychology, we develop models describing the dynamics of users' motivation and behavior studied in psychology, which are influenced by personal values, intervention and social norms. To the best of our knowledge, such a framework incorporating psychology has never been developed and studied from a control perspective. Our work is partially inspired by [15], which studies a social-physical welfare optimization problem depending on prosumers' motives. More precisely, in [15] a DC microgird is considered, where however the prosumers dynamic behavior is neglected and hence no dynamic models describing energy consumption are used. In our paper, we deal with an AC microgrid as physical layer, which has more complex dynamics than a DC microgrid, and dynamic loads influenced by the dynamic behaviour of users. Moreover, in [15] the controller generates inputs only to the microgrid, while the proposed control framework generates also optimal behavioral intervention to affect users' energy consumption in a socially acceptable way. In [5], the behavior of human is modeled by two modes: joining or quitting DSM programs. Different from this binary type of switching model, we let the proposed control scheme reduce users' demand in a "continuous" fashion, and the degree of reduction depends on the motivation (i.e., personal values) of each user and the degree to which each user is influenced by interventions. Additionally, we develop also the control for the considered microgrid, which is not considered in [5]. Paper outline: This paper is organized as follows. In Section II, we develop the overall HCPS framework and introduce the so-called layer equation to deal with its slow-fast dynamics. In Section III we present the control objectives and in Section IV we design a primal-dual controller and analyze the closed-loop stability. A numerical example is presented in Section V, and finally Section VI ends the paper with some conclusions.
Notation: We denote by R the set of real numbers. Given y ∈ R, R ≥y denotes the set of reals no smaller than y. For any w ∈ Z, we denote Z ≥w := {w, w + 1, . . .}. Let 0 and 1 denote column vectors of appropriate dimensions, having all 0 and 1 elements, respectively. Given a state x ∈ R n with n ∈ Z ≥1 , we letx denote its steady state. Let I denote the identity matrix with appropriate dimension. Given a vector v, let v denote its 2 norm. We let N denote the set of N ∈ Z ≥2 prosumers and E denote the set of E ∈ Z ≥1 transmission lines interconnecting the prosumers. Moreover, let N i ⊆ N denote the set of prosumers physically interconnected with prosumer i in the microgrid, and E i ⊆ E denote the set of the transmission lines connected to prosumer i. Let S i ⊆ N denote the set of the social neighbors of prosumer i.

II. HCPS FRAMEWORK
Before presenting the overall HCPS framework in Section II-C, we will first introduce the AC microgrid model in Section II-A, and then the human motivation-behavior models in Section II-B (see Figure 1).

A. AC Microgrid Model
We consider a low-voltage islanded AC microgrid composed of N prosumers that are connected by E resistive-inductive transmission lines. From a physical point of view, we assume that every prosumer can be represented by a Distributed Generation Unit (DGU), or equivalently a distributed storage unit, including a Voltage Sourced Converter (VSC) and a load (see Figure 2). We let ω 0 = 2π f 0 be the speed of the rotating dq reference frame, where f 0 denotes the nominal frequency of the microgrid. Then, provided that the microgrid is balanced and symmetric, and all clocks of the internal oscillators are synchronized, we apply the Clarke's and Park's transformation to obtain the system dynamics in the rotating dq-frame [18], [19].
Given the notation in Table I, the (physical) dynamics of the of prosumer i = 1, 2, . . . , N, can be expressed as where the d and q subscripts represent the direct and quadrature component, respectively. The resistance R Li can represent the base load of prosumer i or simply the system's (parallel) damping, while I Lgi u li , g ∈ {d, q}, represent controllable current loads, i.e., loads equipped with local control units (e.g., smart home controllers). In particular, I Ldi and I Lqi are constants representing the nominal load currents of prosumer i, while u li : R → [0, 1] is the load control input that will be automatically adjusted by a suitable controller, whose dynamics depend on the model of human behavior, which will be introduced in the next subsection. For example, if u li = 0 or u li = 1 for all t, then the actual current absorbed by the loads of prosumer i is 0 or I Lgi , respectively. Note that due to the dynamic load control input u li , the current absorbed by the load of prosumer i, i.e., I Lgi u li , is generally time-varying. Moreover, in Section V, we will show simulation results where also the nominal load currents I Lgi are time-varying and u li can be greater than 1.
In (1), I dk and I qk denote the current exchanged between prosumers i and j ∈ N i through the line k ∈ E i . The ends of the transmission line connecting prosumers i and j are arbitrarily labeled by "+" and "−". Then, the incidence matrix B for the "labeled" graph is given as if prosumer i is the negative end of the labeled transmission line k ∈ E i , otherwise, B i,k = 0. Suppose that prosumer i is the positive end of the transmission line k, then, the dynamics of I dk and I qk are given by (2b)

B. Human Motivation-Behavior Models
In this subsection, we focus on modeling human activities related to the consumption of energy. This will be beneficial for designing load controllers (e.g., smart home controllers and/or suitable Apps [20]) that automatically mimic human behavior. Specifically, we consider two different cases: case i) without social influence through descriptive norms and case ii) with social influence through descriptive norms. Before presenting our models, we first provide in the following for the readers' convenience the definitions of personal values (egoistic, hedonic and biospheric), motivation, behavioral intervention, social norms and contextual factors.
Personal values reflect general and desirable life goals which are used as guiding principles in people's lives to evaluate actions and situations on. Research has identified a set of universal values, meaning every individual endorses these values to some extend. However, individuals differ in how strongly they endorse each value. The more an individual endorses and prioritizes a value, the more influential this value is for someone's preferences and actions [21], [22]. In case of energy behavior, three values appear of particular relevance: • Egoistic values concern goals to acquire possessions, money and status. Energy behavior is often associated with financial benefits, which may motivate individuals with stronger egoistic values to engage in energy behavior for increasing/decreasing the consumption [9]. • Hedonic values concern striving for pleasure and comfort.
Energy saving behavior may require effort or obstruct people from taking certain (convenient) actions, which is why energy saving behaviors may thwart hedonic values. Accordingly, people with stronger hedonic values are expected to have a weaker motivation to save energy. • Biospheric values reflect goals to care about nature and the environment. Energy savings have clear environmental benefits, which is why stronger endorsement of biospheric values is typically indicative of stronger motivation, and thereby stronger engagement in energy saving behavior [23]. Motivation indexes a person's willingness to perform a behavior, e.g., the stronger an individual's motivation to save energy, the more likely this individual is to engage in energy saving behavior. Motivation is generally influenced by personal values. In the context of this paper, when someone strongly endorses biospheric and/or hedonic values, this individual is likely to experience a motivation to take the corresponding actions [24]. For instance, individuals who strongly endorse biospheric values typically feel a stronger motivation to take actions to save energy, eventually increasing the likelihood they will engage in energy saving behavior.
Behavioral intervention includes subsidies and financial rewards that can promote energy savings/consumptions by making it more attractive. The impact of financial incentives is likely more pronounced for individuals with stronger egoistic values, as such individuals care relatively much about money and possessions [9], [23]. Other examples of intervention are persuasion, social support and legal regulation [12].
Social norms in this paper represent the perception that others in a group/community intend to save energy. Social norms can motivate people to engage in sustainable energy saving behavior [10], particularly when the majority does engage in such behavior [11].
Contextual factors in this paper refer to situations that can decrease/increase behavioral 'cost', and hence motivate/demotivate people to act upon the corresponding values. For example, in some situations, pro-environmental actions (e.g., use less air conditioners in the summer) are generally associated with very high behavioural 'costs', e.g., convenience and comfort. As a result, people are less motivated to act upon biospheric values in those situations [25]. Now, we present the proposed human models related to the use of energy.
Case i) For the dynamics of prosumer i, we propose the following second order dynamic systeṁ where b i and m i are the states describing human behavior and motivation, respectively; and s i is the "control input" of (3) representing behavioral intervention and it will be designed in Section IV (see (19) and (21)). All the other variables and parameters in (3) will be explained in the following (see also  Table II). Also, we observe that for a givens i , the steady-state of system (3) satisfies Finally, the compact form of (3) can be written aṡ where A, H, C and D are diagonal, e.g., A = diag(a 1 , . . . , a N ). We interconnect the human layer (3) and the physical layer (1), (2) through the cyber layer, i.e., through the design of an optimal primal-dual controller (see Sections III, IV and also Figure 1). Specifically, the human and physical layers are interconnected through the proposed behavior-based load control, i.e., u li = b i . Detailed explanations about the variables and parameters of system (3) are given in the following.
State and control variables: a) b i : By choosing u li = b i , this quantity represents the degree of supply of the load demand, with 0 ≤b i ≤ 1 for a non-negative s i . For a negative s i , b i can be greater than 1, i.e., energy consumption is promoted by interventions, e.g., making charging plug-in electric vehicles cheaper when the microgrid overall generation/consumption is larger/smaller [26]. More precisely, it is the control input that automatically adjusts the current load demand I Lgi b i of node i by mimicking the behavior of prosumer i. The closer b i is to 1, the more satisfied the load demand of prosumer i is. This is a key variable that plays the role of interface between the physical layer (1), (2) and the human one (3). b) m i : Motivation, with 0 ≤ m i ≤ 1. From (3b), the solution m i can be expressed as follows with 0 ≤ m i (0) ≤ 1 andm i in (4b). If we omit in (3a) the term −h i s i (see the following items for the meaning of h i and s i ), m i can be considered as a "reference signal" for the behavior b i in absence of behavioral intervention (s i = 0) or when behavioral intervention has no influence on prosumer i (h i = 0). Thus, h i s i = 0 implies b i → m i when time approaches infinity. This essentially describes the phenomenon for which the motivation m i acts as a guide for the behavior b i . c) s i : Behavioral intervention, withs i ≤m i /h i . We requirē s i ≤m i /h i since it prevents behavioral intervention from being unreasonably large at the steady state. The variable s i is the "control input" to the behavior of prosumer i. Since h i is nonnegative (see the following items), s i ≥ 0 represents behavioral intervention that can motivate people to save energy and s i < 0 for increasing energy consumption. For a givens i andm i , it follows from (4a) that a larger value ofs i leads to a smaller value ofb i . For instance, this can describe the phenomenon for which major behavioral intervention generally leads to more energy savings. From (3a) and (4a), one can observe that also motivation besides behavioral intervention influence individuals' energy behavior.
Constant parameters: d) The item h i indicates the degree of influence of behavioral intervention s i on the energy behavior b i . In particular, as mentioned above, in case of intervention corresponding to financial benefits, it is reasonable to consider h i = v i . Case ii) Differently from case i), now we consider also the influence of social norms, which may motivate people to engage in sustainable energy-saving behavior. Then, the dynamics in (3b) are replaced bẏ where m j with j ∈ S i denotes the motivation of the social neighbors of prosumer i, and b ij ∈ R ≥0 represents the weight of the social influence. Then, the compact model for the case including social norms is given bẏ where L is the Laplacian matrix associated with the social network, which in this paper is represented by an undirected connected graph. Note that the social network topology is not necessarily identical to the physical topology of the microgrid. For the sake of exposition, in the following sections we will mainly focus on case i). Then, we extend the results to case ii). We assume all the parameters of systems (5) and (8) to be constant over the considered time windows. In order to capture the effect of changes in these parameters both on the transient and steady state, in Section V we will also show simulation results where some of these parameters change during the considered time window.
Remark 1: The proposed models are partially inspired by and consistent with some studies on opinion dynamics. First, they are partially consistent with some opinion dynamics models in which the "topics" (state variables) are possibly more than one and logically correlated [14]. Indeed, the proposed models have two correlated topics, i.e., b and m, with the topic m affecting the topic b. Second, the dynamics of m in (7) are also partially consistent with the continuous-time Friedkin-Johnsen model [13], where an individual is stubborn about his/her opinion m i (depending on v hed and v bio ) and may also be influenced by others' opinion.

C. HCPS Framework and Layer Equation
Recalling from the previous section that the physical layer (1), (2) and the human layer (5) (or (8)) are interconnected through the proposed behavior-based load control, i.e., u li = b i , we have Note that (9) is a slow-fast system in which (9a)-(9g) correspond to the fast dynamics and (9h) corresponds to the slow ones [27]. The time scales of them can be significantly different, e.g., the transient response of (9a)-(9g) can occur within some milliseconds or seconds, and the one of (9h) can be days, weeks or longer. Hence, the slow-fast system (9) can be decomposed into two simpler subsystems, the slow and the fast ones, which allow for a more thorough understanding of the interplay between the slow dynamics of the human motivation, and the fast dynamics of the microgrid (including the behavior-based load control). In the context of our paper, we are interested in how human behaviour influences the microgrid (e.g., voltage, demand and current generation). Thus, it is reasonable to transform (9) into the so-called Layer Equation, which allows us to inspect the fast dynamics [28]. In a Layer Equation, the state of the slow dynamics is assumed to experience little changes in the time scale of the fast dynamics, hence the "slow state" is assumed to be constant in the time scale of the fast system dynamics [27]. The corresponding Layer Equation of (9) is given by where , m can be considered constant. In principle, the smaller t k+1 − t k , the better the approximation of (10)-(11) to (9). Specifically, (10) and (11) represent fast and slow dynamics, respectively. By (10), one can inspect the dynamics of the microgrid for each [t k , t k+1 ).
As t k → ∞, the long-term influence of human values on microgrids can be visible, as will also be shown by simulation. Similarly, one can obtain the corresponding Layer Equation considering case ii) by replacing (11c) with (8b).
In practice, some problems need to be investigated in short and/or long time windows. The problems related to short term performance (with a time scale of milliseconds or seconds) of microgrids have been extensively studied, e.g., in [17], while the problems regarding microgrids in long term have also been investigated, e.g., in the papers [29], [30], in which "long term" refers to a time scale ranging from half to several minutes. In the field of energy planing, some problems are analyzed from the perspective of long and short time windows, where the long time window refers to a decade and short one refers to one year [31], [32]. As an inter disciplinary work, our system model (e.g., (9) and (10)-(11)) fills in the gap between the two cases, i.e., we are able to study the control problem regarding the part of microgrids in a short time window, and study the overall problem regarding the human-in-the-loop system in a longer time window. Although one can study the control of microgrids and human activities separately as in conventional studies, our model provides useful control-oriented insights about the interplay between the physical system and the human in both short and long time horizons, e.g., inspecting the real-time electrical signals influenced by human activities and predicting future energy behaviors. As will be shown in the next two sections, the system (10), (11) is the foundation for the formulation of a social-physical optimization problem and the design of a primal-dual controller to solve it, which constitute the cyber layer of the proposed framework.

III. OPTIMIZATION PROBLEM
In this section we formulate the optimization problem associated with (10), (11), taking into account both physical and social aspects.

A. Physical Welfare
In this subsection, we introduce the control objective of voltage regulation and a cost function associated with the minimization of current generation.
We consider the regulation of V d and V q . Let V r g = [V r g1 , . . . , V r gN ] T ∈ R N ≥0 be the vector of voltage references for V g , with g = d, q. We assume that V r d and V r q are provided by an upper control layer. We would like to achieve exact voltage regulation at the steady state, i.e., lim t→∞ V di (t) = V r di and lim t→∞ V qi (t) = V r qi for i = 1, 2, . . . , N. Furthermore, we assume that the cost function associated with the generation of current is given by where π ci > 0 represents the unit cost of current generation of prosumer i. Given any constant V r d , V r q ands, and assuming thatV d = V r d andV q = V r q by control, the steady-state of the microgrid (10) in the interval [t k , t k+1 ) satisfies where have been substituted in (13). Note also that the steady-state of the microgrid in (13) depends on can be obtained from (11). Overall, from (13) and ξ(t) = m(t k ), one can verify that given (ξ,s,ū d ,ū q ), the forced equilibrium of the Layer Equation (10)-(11) exists and is unique.

B. Social Welfare
In this subsection, we formulate the social (and economic) welfare problem within the considered microgrid.
First, let I Li := (I 2 Ldi + I 2 Lqi ) 1/2 . Then, we assign to every prosumer a strictly concave quadratic 'utility' function U i (b i ) depending on the load satisfaction of prosumer i. Then, the overall utility can be expressed as Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.
where the parameter π ui ∈ R ≥0 weights the satisfaction of the load demand of prosumer i. For example, a relatively large π ui corresponds to a relatively large request of comfort from prosumer i (see [15] for suggestions on how to select the parameters π ui ). By minimizing −U(b), we aim at satisfying prosumers' load demands as much as possible by makinḡ b close to ξ in [t k , t k+1 ) (i.e., making ξ i −b i close to 0). We would also like to guarantee that the intervention s is close to a given reference value s r ∈ R N . In this paper, we let s r = [s r 1 , . . . , s r N ] T be a constant vector, which can represent, e.g., the recommended intervention provided by a higher-layer control system or external parties such as the government or energy provider. Accordingly, we aim at minimizing the cost function s − s r 2 . In simulation, we will also show the effectiveness of time-varying s r .

C. Social-Physical Welfare
Considering the physical and social welfares in Sections III-A and III-B, respectively, we now formulate the overall social-physical welfare optimization problem. Let the optimization variables be denoted by the superscript * and letx c : be the vector of the optimization variables. Then, in view of the equalities in (13), we consider the following convex minimization problem: In (16), α, β, γ , δ and η are positive constants that can be chosen to prioritize one objective over another. The terms depending on u * d and u * q concern the minimization of the control efforts. Note that the supply-demand balance can be adjusted by tuning the parameters α, β, γ , δ and η. For example, if one would like to achieve a supply-demand balance such that the prosumers' demand is satisfied as much as possible, then α needs to be selected relatively "large" with respect to the other parameters. Furthermore, we note that in addition to the equality constraints (15b)-(15f), inequality constraints (see, e.g., [16]) may also be considered for instance to avoid line congestion or too high incentives that are financially unsustainable and undesirable, which will be considered in the future. Note also that, from (13a)-(13b) and (15d)-(15e), it follows that when the optimization problem (15) is solved,

IV. CONTROLLER DESIGN AND STABILITY ANALYSIS
In this section we complete the development of the cyber layer by designing a primal-dual controller that solves the optimization problem (15). Then, we analyze the stability of the overall HCPS framework.
Based on the KKT conditions in (18) and under the assumption that each controller can exchange information among its neighbors through a communication network with the same topology as the physical network, we design the following distributed control scheme by using the primal-dual dynamics of the optimization problem (15), i.e., Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.
where τ b , τ td , τ tq , τ ud , τ uq , τ s , τ a , τ b , τ c , τ d , τ e ∈ R N×N are positive diagonal matrices, which can be tuned to adjust the controller response. Moreover, the vectors φ A , φ B and φ C are the controller input ports, which will be used in the next subsection to interconnect the controller (19) with (10) and (11). We also note that since in the optimization problem (15) the objective function (16) is quadratic with respect to b * , I * td , I * tq , u * d , u * q , s * and the constraints (15b)-(15f) are linear, then, given constantφ A ,φ B andφ C , it can be shown that the solutionx c to (19) is unique.

B. Stability Analysis
In this subsection, we will show that (10) and (11) in closed loop with the primal-dual controller (19) are stable and converge to the solution of the optimization problem (15).
The dynamics in (10g) can be written aṡ with −A being Hurwitz. Thus, given any diagonal and positive definite matrices Q 1 , let P 1 denote the corresponding unique solutions of the Lyapunov equation where P 1 is a positive definite matrix. Then, we interconnect the controller (19) with (10), (11) by choosing Note that the controller (19), the interconnections (21)- (22) and the communication network for exchanging data form the cyber layer. By (22), one can see that to control the HCPS as shown in Figure 1, the controller (19) needs to collect I td and I tq from the physical layer (1), (2) and collect b from the human layer (5). Meanwhile, the controller generates control inputs u d and u q for the physical layer and s for the human layer. Then, it is clear that the cyber layer interconnects the physical and human layers as shown in Figure 1. Let (10). Now we are ready to present the main result of this paper. Theorem 1: The closed-loop system (10), (11), (19), (21) and (22) converges to an equilibrium solving (15).
Proof: We take three steps to conduct the proof.
Step 1: In this step, we first propose the following storage function [34] for the physical system corresponding to (10a)-(10f) as which satisfieṡ along the solutions to (10a)-(10f). Now, supposing without loss of generality that the loads absorb positive reactive power (i.e., the loads are predominantly inductive rather than capacitive), I Lq is negative definite. Then, by virtue of the Young's inequality [35], , with ζ 2 and ζ 3 being arbitrary positive reals.
Then, for the dynamics in (20) with ξ in (11), we propose the following storage function S h =ḃ T P 1ḃ , which satisfieṡ S h = −ḃ T Q 1ḃ − 2ḃ T P 1 AHṡ * , along the solutions to (20).
Then, for the overall system (10), the storage function S ph := S p + S h satisfieṡ along the solution to (10). Recalling that I Ld and I Lq are bounded, we observe that the terms in the first and second lines can be made nonpositive by selecting sufficiently large ζ 2 and ζ 3 , respectively. With the selected ζ 2 and ζ 3 , also the terms in the third line can be made nonpositive by selecting a sufficiently large Q 1 . Then, we haveṠ ph ≤İ T tdu * d +İ T tqu * q − 2ḃ T P 1 AHṡ * , which implies that the system (10) is passive with respect to the supply rate [İ T tdİ T tq −2ḃ T P 1 AH] [u * T du * T qṡ * T ] T and storage function S ph .
Step 2: In this step, we propose for the primal-dual controller (19) the following storage function which satisfiesṠ c ≤ −u * T dφ A −u * T qφ B −ṡ * Tφ C along the solutions to (19). This implies that the controller (19) is passive with respect to the supply rate −[ṗ T (22).
By (6), one can also obtain that m i (t k ) →m i as t k → ∞, which implies that ξ →m as t k → ∞. In other words, the primal dual controller solves the optimization problem (15) withξ =m in (15f) as t k → ∞.
Remark 2: The controller in (19) with φ A , φ B and φ C in (22) can be implemented in a distributed way. A local controller needs to exchange only the information of its voltage references V r di and V r qi with its neighbors' controllers. This is implied by the terms J 3 V r d , J 4 V r q , J 4 V r d and J 5 V r q in (19g) and (19h).
Remark 3: The proposed framework can also incorporate the scenario in which human motivation may be influenced by contextual factors (see Section II-B) on a short time scale, i.e., the parameters c i and d i in (3b) can change depending on the underlying circumstances where people are involved. For instance, in compact form, the matrices C and D in (5b) can change along time as long as they belong to the positive-diagonal-matrix sets C := {C 1 , C 2 , . . . , C σ } and D := {D 1 , D 2 , . . . , D κ }, respectively, where σ, κ ∈ Z ≥1 are finite integers. Thus, the dynamics regarding m are represented by a switched linear system. The influence of contextual factors can be explained by the following illustrative but suitable example: people might be motivated to increase comfort associated with hedonic values when they arrive at home after a stressful day of work even though getting more comfort may increase the energy consumption. The reasoning behind this behavior is that reducing comfort after a stressful day of work can be perceived as more "costly" than usual, due to the specific context of feeling tired after the stressful work day.

C. HCPS Considering Social Norm
In this subsection, we briefly extend the results of Theorem 1 to case ii), which includes the influence of social norms on the dynamics of people's motivations (see (8b)). Thus, the resulting model consists of (10), (11) in which (11c) is replaced by (8b). With ξ obtained by (11) and the fact that −(C + D + L) is negative definite, the controller in (19) is still applicable to the proposed HCPS including social norms.
Then, the rest of the analysis can be conducted analogously to the proof of Theorem 1. Therefore, the following corollary holds.

V. SIMULATIONS
We consider an islanded AC microgrid consisting of ten prosumers with parameters in Table III. The values of the parameters of the human layer will be provided in the text below and we aim as future research to identify these values in a similar way to the one proposed in [15]. In the following, we consider three simulation scenarios. Specifically, in Scenario i), we consider the case consisting of the model of the AC microgrid (1) and the model of human behavior and motivation (3). To show the effectiveness of social influence in energy saving, Scenario ii) additionally considers social norms. That is, the corresponding HCPS consists of the model  of the AC microgrid (1) and the models of human behavior (3a) and motivation (7), and the controller (19) with the ports in (21) and (22). The first two scenarios are meaningful because, given constant nominal loads, we can still observe a dynamical behavior due to human activities. Different from Scenarios i) and ii), where the nominal load is time-invariant, in Scenario iii), we consider a more realistic situation in practice, where the nominal load is time-varying (i.e., I L is considered as a time-dependent external input) and load data are taken from the database of power consumption of New York Central Park [36]. Moreover, differently from Scenarios i) and ii), in Scenario iii) we make the intervention reference s r time-varying as well, in order to encourage prosumers to decrease the power consumption during the peak hours, and increase the power consumption during the off-peak hours. The influence of contextual factors on energy consumption is also considered.  The simulation results for Scenario i) are presented in Figure 4. In each plot of Figure 4, we also show an enlargement of the first 6 hours in order to make the transient dynamics clearly visible. In the enlargements, we observe that the dynamics of the states are piece-wise constant, which is the consequence of the selection of t k+1 −t k = 1h in the Layer Equation (10)- (11). Note that one can choose different time intervals for t k+1 − t k depending on the specific approximation of the demand. In the first plot of Figure 4, we observe that due to the influence of biospheric values v bio , the motivation ξ of prosumers indeed decreases and finally reaches a steady-state value. Due to intervention s i (approximately equal to ∼ 0.2) shown in the forth plot of Figure 4, the power consumption is lower than the amount that they would have consumed in the absence of the interventions. This can be observed from the first plot of Figure 4, where the values of behaviors b i are smaller than the corresponding motivations ξ i , and the reduced value corresponds to s i . Moreover, the obtained s i is very close to the given reference value of intervention s r = 0.2×1 T 10 (see the fourth plot in Figure 4). Now we discuss the performance of the AC microgrid. We show the voltage trajectories in the second plot of Figure 4, where the voltage V d is stabilized around the reference value V r d = 120 √ 2 V. Moreover, due to the "decreasing" motivationbehavior dynamics shown in the first plot of Figure 4, the absorbed current I Ldi b i is time-varying and decreases, and consequently the current generation is also time-varying and decreases as well (see the third plot in Figure 4). Finally, note that we have verified in simulation that V q → 0, but omit the corresponding plot due to page limit.
For the sake of comparison, we present in Figure 5 the simulation results obtained by considering the motivation m to be constant 1 T N with random variations. This is done to simulate the scenario where the influence of human personal values are not taken into account and the reduction of power consumption is purely driven by intervention. First, note that the values of intervention s in Figure 5 are larger than those in the fourth plot of Figure 4 under the same reference s r , which means a larger cost for intervention. More importantly, with a larger cost for intervention, the values of b are still larger than those in the first plot of Figure 4, which implies more power consumption. Indeed, the prosumers in this case consume 2139 kWh more. This reveals that, if one omits the influence of human values, it is highly possible that the government/society estimation of cost for intervention will be higher, and the energy consumption will not be necessarily lower. If we consider b to be constant as well, where the value of b is directly determined by solving the optimization problem, instead of dynamically depending on s, then, one recovers the problem in [15].
Scenario ii) In this scenario, we additionally consider the influence of social norms. For the social network topology among the prosumers, we select L = 0.6 · 10 −4 × BB T in (8b).  The other parameters are as in Scenario i). We show the time evolution of the motivation ξ (in Layer Equation) and behavior b in Figure 6 and omit the other signals due to the similarity to those in Scenario i). Since the Laplacian matrix L in (8b) represents a connected and undirected graph, one can see that the components in ξ and b in Figure 6 converge to values that are closer to each other with respect to those in the first plot of Figure 4 in Scenario i). Similarly to Scenario i), in Figure 6 one can observe that the discrepancy between ξ i and b i is approximately 0.2, which is the consequence of the intervention s i . Due to the influence of social norms, it is possible to calculate that the energy consumption of each prosumer in the considered time horizon of 7 days is reduced by 2.55 kWh/day in average than that in Scenario i).
Scenario iii) Compared with Scenarios i) and ii), where the nominal loads and behavior interventions are time-invariant, this scenario is more realistic and considers time-varying nominal loads, time-varying behavior interventions, and also the influence of contextual factors. Specifically, we simulate the case of time-varying nominal loads (i.e., time-varying I L ), where load data are taken from the database of power consumption of the area of New York Central Park [36]. The simulation results are shown in Figure 7. Furthermore, we consider the influence of contextual factors in order to take into account the effects of human motivation also on a shorter time scale, due for example to the fact that during some hours of the day one can more or less strongly endorse a value rather than another. It is indeed reasonable to expect that due to adversary weather conditions and/or a stressful working day, the influence of the hedonic values on human motivation may be stronger. For this, we choose from 19 to 24 o'clock, which corresponds to the scenario in which the prosumers increase the energy consumption in order to acquire pleasure and comfort, for instance when they arrive at home after a stressful work day. This is shown in the first plot of Figure 7, in which the motivation ξ (dashed lines) from 19 to 24 o'clock is slightly larger than those of other times. Different from Scenarios i) and ii), where we considered a constant intervention reference s r , in this scenario we consider the intervention reference s r to be time-varying during the day, as shown in the fourth plot of Figure 7. Specifically, one can see that in the example we introduce "negative" reference for encouraging the use of energy during the midnight and "positive" reference for encouraging energy saving during the evening. The benefits of time-varying intervention reference s r and the corresponding effect on s can be observed from the second plot in Figure 7, where prosumers shift the time of consuming energy by using more energy during the midnight and less during the day compared to the case of static intervention. Specifically, the actual current load 1 T N I L b (solid lines) is smaller than the current load 1 T N I L ξ that people would have consumed (dashed lines) during the peak hours in the evening, and larger than the current load I L ξ that people would have consumed in the midnight. Thus one can see that, by a suitable choice of s r , it is possible to shift the use of energy from the peak hours to the off-peak ones. If one applies a static s r , it is difficult to shift the use of energy and one still has peak hours (i.e., the evening), though the peak is reduced by the static s r (see the orange line in Figure 8). Finally, from the third plot in Figure 7, we can observe that the voltage V d is stabilized around the reference voltage value V r d = 120 √ 2V. Note that the controller (19) becomes a time-varying system in Scenario iii) instead of a time-invariant system leading to Theorem 1.

VI. CONCLUSION
We have developed a framework to bridge the disciplines of systems & control and psychology to jointly study energy dynamical behavior of human together with the dynamics of the microgrid. Specifically, we have formulated a HCPS framework to describe energy behavior of humans in social networks and also their interactions with the microgrid via a cyber layer. The developed models describing human energy behavior depend on motivation and behavioral interventions, and are consistent with the findings in psychology. With the developed HCPS, we have formulated a social-physical welfare optimization problem and have designed a distributed primal-dual control scheme. It is proven that the proposed control system generates the optimal intervention to humans and control inputs to the microgrid, stabilizing the closedloop system whose state converges to an equilibrium solving the considered optimization problem. The HCPS framework shows that human intrinsic factors (values and motivation) and behaviours can have substantial impact on the energy system and should therefore be accounted for in energy system modelling. The results in this paper have also shown the interplay among human, physical systems, energy consumption and a player providing references for interventions. The developed models of human activities not only interpret the findings in psychology from the control viewpoint, but also are consistent with the studies on opinion dynamics.
Interesting future research includes the design and analysis of HCPS frameworks to embed the proposed human layer into analogous optimization control problems for the main grid (e.g., [16]) or DC microgrids (e.g., [37]). Additionally, different operation modes and control objectives can be considered, e.g., the grid-connected operation mode and power sharing objective. One can also expand the current human motivationbehavior models by including variables/factors from prominent psychological models, such as the value belief norm theory [24] and norm activation model [38].