Multi-Agent-Based Voltage Regulation Scheme for High Photovoltaic Penetrated Active Distribution Networks Using Battery Energy Storage Systems

This paper develops a distributed voltage regulation scheme for high Photovoltaic (PV) penetrated distribution networks by utilizing battery energy storage (BES) units. In this study, multiple BES units form a multi-agent network, in which each BES unit acts as an individual agent and can communicate with its neighbors to perform the distribution network’s voltage regulation in a cooperative manner. To cope with the uncertainties of PV power and load demand, a receding horizon-based approach is proposed, where BES control decisions for regulating bus voltages are updated with the update of the system’s operational conditions (PV power, load, etc.). An efficient solving method – Distributed Alternating Direction Method of Multipliers Algorithm (D-ADMM) is applied to solve the proposed model under a colored network, where the BES agents with the same color synchronously update their states. Case studies are conducted on modified IEEE benchmark systems to validate the performance of the proposed method.


I. INTRODUCTION
A. BACKGROUND Photovoltaic (PV) solar power is one of the most promising renewable energy technologies in the world.The global PV market has been experiencing a tremendous growth.By 2018, the worldwide installed PV power capacity had reached 430GW [1].PV integration can provide local production support to energy demands and relieve power Distribution Networks (DNs)' congestions.Nevertheless, due to its inherent stochastic nature, the integration of PV induces new challenges to the operation and control of DNs.One major challenge is the voltage rise problem that usually occurs in midday hours when solar radiation excesses the local demand.The redundant solar power would lead to rises of The associate editor coordinating the review of this manuscript and approving it for publication was Eklas Hossain .bus voltages, which consequentially would affect the grid's security and even cause physical damages to equipment [2].

B. RELATED WORK
To prevent the voltage rise problem, a variety of solutions have been proposed [3]- [5], including: (1) upgrade the grid, which is usually expensive and labour-intensive [3]; (2) utilization of voltage regulation devices such as fixed/switched capacitor banks and load tap changers [4].Most of these devices are designed to operate in hourly basis time scale, and thus can hardly provide fast response to sudden changes of PV power [5]; (3) PV power curtailment, which is not preferable due to the waste of renewable energy [6].
Considering the high resistance/reactance (r/x) ratio in DNs, voltage magnitudes are sensitive to variations of active power.Therefore, an alternative solution for alleviating the voltage rise problem is to increase power consumption to accommodate the surplus solar power at some time intervals.VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License.For more information, see http://creativecommons.org/licenses/by/4.0/ This can be implemented through load management or using energy storage systems.Energy storage technologies have been rapidly developing in recent years.Currently there are different kinds of available energy storage techniques, including battery, fly-wheel, pumped hydro systems, compressed air energy storage, electrochemical double-layer capacitors (EDLCs), and so on.In the current market, compared with other energy storage systems, battery energy storage systems (BESS) is one of the most cost-effective ones due to its advantages of fast response time and high ramp rates.BES systems can be utilized to store excess PV energy and release it when there is a mismatch between power supply and consumption [7].Numerous control strategies have been proposed in literature to deal with voltage fluctuation problems in DN, which can be classified into three types based on their architectures: 1) centralized, 2) decentralized; 3) multi-agent systems (MAS) based distributed control [8].Centralized control techniques have been developed for controlling multiple BES units to regulate voltage in DNs in [6], [9].However, centralized control scheme requires high investment in communication structure, which is vulnerable to external attacks and might cause information loss with limited bandwidth [10].The second category is decentralized control method which only uses local information.References [11], [12] presented a coordinated control of PV and BESS for voltage regulation based on less complex communication infrastructure.However, due to the limited information sharing, it might induce frequency and voltage deviations which affect power quality [13].
Recently, MAS based distributed control for voltage regulation has attracted interests in academia [14]- [20].Reference [15] proposes a two-stage voltage control scheme of dispersed DGs in a DN using the distributed Lagrangian primal dual sub-gradient algorithm.Reference [16] addresses the voltage regulation problem through the establishment of a rank-constrained semi-definite OPF model and a distributed solving approach.Reference [17] proposes a coordinated voltage control scheme in a low voltage distribution system, where a consensus algorithm is adopted for ESS active power output control.Reference [18] develops a hybrid method that combines both local droop control and distributed control to regulate the bus voltage profile.The method facilitates the effective operation of BESs under different conditions.In [19], a MAS based control strategy is developed to coordinate energy storage systems in a micro-grid, where the energy storage agents are organized into different groups to cooperatively regulate the micro-grid's voltage and frequency.In terms of considering communication time delays, [20] presents a distributed fixed-time MAS distributed control for BES and PV systems to perform voltage regulation and frequency restoration.However, most of above works [15]- [20] require all-to-all information exchange of agents; this can only be achieved either via a central controller or through a fully connected network topology, which limits their practical applications.
Moreover, aforementioned methods often perform realtime control in individually discrete time steps, and does not consider the coordination of control actions on a horizon covering multiple time steps [21].This would often lead to solutions that are far away from optimal over a finite horizon.Further, existing works often do not consider the impact of frequent charging/discharging actions on BES's lifetime.In addition, most of these works do not consider the impact of forecasting errors of PV power and load.

C. CONTRIBUTIONS OF THIS PAPER
This paper proposes a MAS based control scheme for controlling BESs to regulate voltage for a DN with high PV penetration.The major contributions of this paper include following aspects: 1. Propose a voltage regulation model based on the cooperation of networked BESs.The l1-norm regularization is embedded in the model [22], [23] for finding the minimum number of BES units and minimum actions to all BES units to accomplish the voltage regulation task; 2. Propose a receding horizon strategy [24] to control the BESs to regulate voltage.The proposed strategy updates the system's operational condition with proceeding of time; based on this updated information, the control actions to the BESs are updated.In this way, the negative impact of the uncertain variables in the system's operation environment can be mitigated to a large extent; 3. Develop a D-ADMM [25]- [27] based approach for solving the proposed voltage regulation model.In the developed approach, voltage regulation is realized through limited information exchange among BES agents in a fully distributed manner.
The paper is organized as follows.Section II outlines the DN architecture and the multi-agent system.Section III presents the formulation of the voltage regulation problem.Section IV presents the distributed solving approach based on the D-ADMM algorithm.The effectiveness of the proposed method is examined via case studies in Section V. Section VI concludes the paper.

II. MULTI-AGENT SYSTEM STRUCTURE A. SYSTEM STRUCTURE
Figure 1 depicts the conceptual architecture of a DN integrated with multiple BESs and PV generation sources.In this scenario, each BES unit is controlled by an Energy Management System (EMS) and acts as an independent agent.There is an underlying communication network that facilitates the communication among the BES units.

B. NOTATIONS AND NETWORK MODEL
Considering a DN integrating with H controllable BES agents that can communicate with each other.The DN can be modelled as an undirected graph G = {V, ε}, where V = {1, 2, • • • , H } denotes the set of BES agents and ε ⊆ V × V is the set of edges.An edge connecting pairs of BESs  that can communicate with each other.An edge (i, j) (i, j ∈ V) connects BES agents i and j.The BES agents connected with the ith BES agent is known as the ith BES's neighborhood, whose set is denoted as H i = { j : (i, j) ∈ ε}.The degree of BES agent i (denoted as D i ) is defined as the cardinality of We consider that the BES agents in the network are colored by L different colors, such that two neighboring BESs never share the same color.The first C 1 BES agents are assigned with color 1 and are denoted as the set of The rest BES agents are colored in a same way.C l BES agents marked in color l are counted denoted 2 illustrates an example of a colored network, which comprises 7 agents and 12 edges.The agents are assigned with 4 different colors.The colored agents are divided to following four groups:

III. VOLTAGE REGULATION PROBLEM FORMULATION A. MODELING OF PV AND LOAD UNCERTAINTIES
Voltage regulation tasks in DNs are usually based on prediction of PV solar power and power demand.The prediction models used in this study are presented as below.

1) SOLAR POWER PREDICTION MODEL
Assuming PV array is equipped with a MPPT controller.The solar power generated by PV array are predicted as [28]: where P PV (t) denotes PV output power at time t; P PV _r is the rated output power of PV array, f PV is the de-rating factor considering shading, wiring losses and snow cover, etc. G STC and T STC are the solar radiation and temperature on PV array under standard conditions.G (t) and T (t) are the solar radiation and temperature in current time, and α T is the temperature coefficient of power.G (t) represents the predicted solar radiation at time t, calculated as: where G (t) denotes the estimated solar irradiance; µG (t) represents the estimation error that is assumed to follow a zero-mean Gaussian distribution.

2) POWER DEMAND PREDICTION MODEL
The predicted power load P load (t) at time t (kW) is calculated: where Pload (t) and δ load (t) are the estimated load value and estimation error.The estimation error is assumed to follow a truncated normal distribution with zero mean and the standard deviation of 5% [29].

B. RECEDING HORIZON BASED VOLTAGE CONTROL
PV power prediction accuracy would decrease with the increase of the prediction time horizon.To better handle the unavoidable prediction error, a receding horizon approach is applied in this study for voltage regulation.That is, in the first time step, the solar and load data are predicted for the while horizon (i.e.T time steps).Then, a voltage regulation model covering all the remaining T time steps is formulated (will be presented in Section IV) and solved.Only the control actions of the first T time steps are actually applied to the BESs.When the time proceeds to the T + 1th time step, the solar & load prediction is re-performed for the next T time steps, the voltage regulation model is re-formulated and solved, and the control actions of the next T time steps are applied.This process repeat until the end of the whole horizon is reached (Figure 3).In each round of the receding horizon, a voltage regulation model is formulated and solved.The model determines: (1) which BESs from the BES network are selected to control; VOLUME 8, 2020 and (2) how much power are charged/discharged to them.The objective function of the voltage regulation model is represented as Eq. ( 4).It aims to find minimum number of BES units and minimum actions of all BES units while satisfying the voltage constraints over the whole control horizon.
The decision variable P i refers to the ith BES's power output; P i = P ch/dis,i (1) , P ch/dis,i (2) , . . .P ch/dis,i (t) T ∈ R T .P ch/dis,i (t) refers to the charging/discharging power of the ith BES at time t.H is the sets of buses with BESs installed.Eq. ( 5) represents the l1-norm expression of the objective function, which ensures sparsity of P i (i.e. with least numbers of nonzero entries).
Model ( 4) is subjected to following constraints, (1) BES operation constraints: where T and t are the total number of time steps over the whole control horizon and the duration of each time step, respectively; E i (t) is to the ith BES's energy level at time t; P bat i,max denotes the ith BES's rated power; E i,min and E i,max are lower and upper limits of ith BES agent energy capacity; η ch and η dis refers to the BES's charging/discharging efficiency, respectively.Constraint (6) restricts the ith BES's power output cannot exceed its power capacity.Constraints ( 7)-( 8) restrict the energy stored in the ith BES must be within an allowable range, so as to protect the BES from over charged/discharged and prolong its lifetime.
(2) Bus voltage constraints: where N denotes the set of system buses; v n,min and v n,max are lower and upper voltage limits for bus n; Eq. ( 9) denotes the voltage constraints for all the system buses n at time t.
(3) Power balance constraints: where M is the sets of buses with PV sources.It is assumed that the DN operator can purchase electricity from the external grid.P ext (t) denotes the purchased power at time t.(4) PV power constraints: where P PV _r,i denotes the rated power capacity of the ith PV source.

C. SENSITIVITY MATRIX
The influence of BES power outputs and bus voltage are calculated as below.Firstly, the relationship between power output and voltage can be expressed through Jacobian sensitivity matrix derived from the Newton-Raphson power flow calculation [30].More specifically, the general conversion form between small variations in the system's active and reactive power output ( P and Q) and the variations in phase angles and voltage ( θ and V ) profile can be described as: Due to the high r/x ratio value in DN, changes in active power injections might significantly affect bus voltage variation (reactive power Q injection has little effects and thus can be neglected).The voltage value at a time step can be approximately calculated based on the voltage at the previous time step, the Jacobian sensitivity matrix, and the active power output change P: Generally, a DN system's bus voltage variation is set to be within ±5% of the nominal value.Once the voltage of one or more buses fall out of the allowable range, the DN operator would need to bring voltage back to the secure range as soon as possible.However, in practical situations, the voltage is hardly to be regulated back to the secure rage in the first time.Therefore, a relaxation factor ξ is applied to all the system buses [31], to let the bus voltage can be regulated smoothly back to the secure range by the end of the control horizon T .: where ξ = ξ T is the vector of slack variables.

IV. D-ADMM BASED SOLVING APPROACH
Considering the distributed nature of energy resources in DNs, it is impractical (or at least very difficult) for a controller to dispatch the large number of dispersed energy resources centrally.Thus, a MAS based distributed control scheme would be preferable for DN's voltage regulation.
In this study, we develop a distributed control strategy to solve model ( 4)- (14).The control strategy is based on limited information exchange among neighboring BESs using specific communication protocols that were designed for parallel processing.

A. PROBLEM FORMULATION
The proposed model in Eq. ( 4)-( 14) is the sum of separable convex objective functions with linear constraints.
All constraints ( 6)-( 11) and ( 14) are copied to each BES.P i = (P 1 , . . ., P H ) represents the decision variables.Model ( 15) is solved in an iterative manner; in each iteration, each BES has individual objective function, and sends a copy of its solution to other BESs in the network.We use P i ∈ R n to denote the copy hold by the BES i, which represents the solution of the whole BES network at the current iteration.f i (P i ) refers to the separated local objective functions for the ith BES agent.P i ∈ P i represents the feasible local variable set P i for BES agent i. BES i only can access to its private objective function f i and private set P i .P i = P j for all BES agents i and j with edge i, j ∈ ε, which enforces all copies to be equal since the network is connected.
As presented in Section II.B, we assume the network topology is colored by L colors.BES agent s with color l are marked as C l .There are colored sets C 1 , C 2 , . . ., C l in total, with the cardinality of C l = |C l |.Here {C l } L l=1 partitions V. Therefore, problem (15) can be further converted as: ) is copied to all the BESs with color l.There is , where D is the arc-incidence matrix of the network topology; I n is the identity matrix of dimension n; ⊗ denotes the Kronecker product.Each column of D is associated with an edge (i, j) ∈ ε.The ith and jth entry of D is with the value of either 1 or −1, and the rest entries are 0.

B. MAS VOLTAGE CONTROL BASED ON D-ADMM
Model ( 16) can be reformulated in the way of putting the decision variables of the same-colored BESs together to make the model be suitable for distributed implementation.Given L functions g l , L sets P l , and L sets of the full column rank matrix A l : where P l = (P 1 ,. . .,P L ) is the optimization variable; A l = D T l ⊗ I n is the transported node-arc incidence matrix of the graph G.Then, the original problem (model ( 4)-( 14) is transferred to a color-based form.With this formulation, D-ADMM can be applied to solve the model.Firstly, model ( 17) is rewritten to be the form of the augmented Lagrangian function with penalty terms: Here λ is the dual variable and ρ is a positive parameter.Using the D-ADMM algorithm, an iterative procedure can be applied to find the optimal value of P l and λ: where i = 1, 2, ..., L; k denotes the iteration number.
For BES agents with color l, the optimization problem of ( 19)-( 22) is decomposed into C l optimization problems that can be solved in parallel.In this way, the variable vector P l = (P 1 , P 2 , . . ., P L ) is updated as: The first two terms of Eq. ( 23) are linear combinations of the Lagrangian and relative to the agents with color l.The last term in Eq. ( 23) can be expressed as: Because color-l agents are not connected with each other, A T l A l is a diagonal matrix, in which the diagonal elements being the degree of the respective agent.Then, Eq. ( 24) can be simplified as: where λ k A l P l = i∈L l j∈H i λ k ij T P i , λ ij is defined for i < j and associated with constraint P i = P j in (17).The last term in (25) does not depend on P l , and thus can be ignored.Then, Eq. ( 23) can be simplified as: where The auxiliary dual variable is expressed as: Then problem ( 27) is decomposed into C l sub-problem and can be solved in parallel: After the updates of P i and dual variable γ k i of BES i, there is: In the computation process, the ith BES receives charging/discharging information from all its neighbouring BESs with lower-order colours.After that, ith BES solves model (28); meanwhile, the BESs with the same colour update P i individually.In this way, the parallel implementation improves the convergence speed.

C. OVERALL WORKFLOW OF MAS BASED VOLTAGE REGULATION
By applying the D-ADMM and receding horizon strategy, the overall showing in Eq. ( 18) and ( 26)-( 29), the overall voltage regulation procedures are shown as follows: It should be noted that, in the each optimization round of the receding horizon: 1) all the initial input data and system dynamic constraints must be adjusted; and (2) the dual variables should be set with zero at the end of the optimization.
Remark: With the communication topology of the BES network presented before, the D-ADMM algorithm is proven to strictly converge in [23].

V. CASE STUDY A. SIMULATION SETUP
In this section, we report the results of the numerical simulation conducted for evaluating the performance of the proposed method.The simulation is implemented by YALMIP toolbox and MATLAB (version R2018a), and executed on a 3-GHz DELL PC with Intel i7-4790 3.60 GHz CPU and 8 GB RAM.We use the load and solar power data recorded in a typical summer day in south Australia [32], [33].PV penetration level is set to be 50% during the peak load hours.[for l = 1, 2, ...L] do 6: [for i ∈ C l in parallel] do 1) Update auxiliary dual variable according to (27).
2) BES Agent i with color l receives the variables from j ∈ H i , update optimal variables P k+1 i i∈H according to (28).Send updated value P k+1 i to its neighbors.7: End 8: End 3) For all L ∈ V, update the dual variables according to (29) to obtain γ k+1 i .10: End 11: II.Set k = k + 1 and return to Step I until a predefined stopping criterion is met.12: End Output: The entire time horizon is set to be 2 hours (11pm-13pm) with the duration of each time step is 5 minutes (i.e.t = 5 min).The control horizon is 15 minutes.Hence, T = 3, T = 24 time steps.P i ∈ R 24×1 , where P i (t) refers to the power of ith BES at the ith time step.
The simulation results of the proposed method in each iteration are compared with the optimal solutions, with the comparison difference calculated as: where k is the iteration index; f i P k i and P k i are objective function value and solutions of the D-ADMM control scheme, respectively; J i and x * denote the objective function value and solution obtained from the centralized optimization with receding horizon.

B. 15-BUS SYSTEM
We test our method on the modified IEEE 15-bus system, whose topology is shown in Figure 4.The network parameters of the system are available in [34].The secure range of the voltage in each bus (excluding slack bus) is set as [0.95, 1.05] p.u. Two PV panels are located on buses 8 and 10 respectively; they can operate up to 0.9 power factor.Five BESs are installed at bus 4, 7, 9, 11, and 14.The relevant BES data are given in Table .1 [35].We firstly analyze the simulation based on the PV and load profiles without considering forecasting errors.Figure 5 shows system voltage profile over the entire time horizon in buses 7-10.It can be observed that the voltages significantly violate the system voltage constraints at t = 10 and t = 15.With the relaxed voltage constraints, the system operator can mitigate the voltage rise issue smoothly through stepwise voltage regulation.As long as all the monitored bus voltage profiles are brought back to the secure range, the system does not further require any supports from the BESs. Figure 6 shows the detailed SOC profiles of all five BES agents over the entire control horizon.
Table 2 reports the BES power outputs under different values of the relaxation factor ξ .The relaxed voltage upper limits are 1.053 p.u. and 1.0515 p.u., and the voltage constraints  are decremented by 0.001 p.u. and 0.0005 p.u. per time step in three time steps, respectively.It can be seen that when the value of ξ decreases, the involved numbers of BES agents increase and the amount of charging/discharging power increases as well.The relaxed voltage constraints provide proper correction of the voltage profiles especially under severe conditions.This allows the DN operator to adopt most suitable control actions to achieve a trade-off between security and effectiveness.
To analyze the convergence of the proposed method, Fig. 7 compares ε (k) and x diff (k) of the results from D-ADMM and centralized optimization using the interior-point method, where the x-axis and y-axis represent the iteration numbers used to reach the same accuracy (pre-defined as 0.01) and the error of the objective function, respectively.The stopping criteria is set as 0.01.The proposed D-ADMM control scheme converges after 21 iterations, with the total time consumption of only 1.27 seconds.The results also show that through limited information exchange between the neighboring BESs, the same global optimized solution can be obtained by both centralized and distributed approaches.
We further compare the convergence performance of the D-ADMM based voltage regulation with the original ADMM based approach, and the results are shown in Figure 8, We can find that both of the two algorithms can guarantee the optimality of the solution, but the convergence speed of D-ADMM (21 iterations) is much faster than that of ADMM (41 iterations).

C. RESULTS ANALYSIS WITH PREDICTION UNCERTAINTIES
In this sub-section, effectiveness of the proposed receding horizon strategy on voltage regulation is evaluated, with taken into account the prediction errors of solar power and load.The load demand and solar radiation are predicted 2 hours ahead and the prediction is updated every 15 minutes.Figure 9 shows the predicted and actual profiles of load and solar irradiance respectively.
Figure 10 shows the BES power output profiles with and without using the receding horizon control approach.The red line indicates the control action plan without the receding horizon control approach; only the actions during the first 15 minutes are actually applied.The blue dash line denotes the corrected control scheme based on the receding horizon control.It can be observed that the BESs adapt to the updated environmental variables.With the proposed  receding horizon control algorithm, the BESs output power are smoothed along whole control horizon.For example, for 1th, 3th and 5th BESs, The BES power output is smaller     In this case, we test our algorithm on a 43-bus system.The topology of the 43 bus is shown in Figure 11, where four PV panels are installed at buses 7, 13, 19, and 41.Eight BES units are installed at buses 9, 3, 11, 17, 26, 6, 39, and 40, respectively.Same with the previous case, the entire control horizon and the duration of each time step are set as 2 hours and 5 minutes, respectively.
Similarly, Figure 12 and 13 present the comparison of BES power output and the objective function value between the D-ADMM and centralized optimization approaches.From the results, we can find that each BES unit communicates with their neighbors and finally reached the optimal decision cooperatively.The decision variables for all BESs converges after several iterations and the objective function value reaches the optimality, same with that obtained by the centralized optimization.

VI. CONCLUSION AND FUTURE WORK
This paper proposes a MAS based voltage regulation framework in DNs with high PV penetration.The method controls the BES units' power output to mitigate the voltage rise issue in a distributed manner.To achieve this, the l1-norm is embedded in the objective function.The proposed method is mainly characterized by two features: (1) it uses a receding horizon approach to continuously update the system's states and the control actions to the BESs.The simulation results on two modified IEEE benchmark systems show that the receding horizon strategy can well adjust the BESs' outputs while maintaining the bus voltages within the secure range; (2) it uses a fully distributed control scheme based on D-ADMM algorithm for determining the charging/discharging control actions to the BESs.The simulation results show that the D-ADMM algorithm has a strong capability for performing global optimization and has faster convergence ability than the original ADMM algorithm.
Future works can be conducted in several directions.For example, a more comprehensive system structure can be studied by considering various kinds of voltage regulation devices such as inverter and capacitor banks.In addition, the penetration of reactive power optimization can be also considered for voltage regulation.

FIGURE 1 .
FIGURE 1. Conceptual architecture of the radial DN with PV and distributed BES units.

FIGURE 2 .
FIGURE 2. Example of colored MAS based framework.

FIGURE 4 .
FIGURE 4. Communication network of modified 15-bus distribution system with five BES units, seven edges labeled in three colors.TABLE 1. BES units technical data.

6 .
BES SOC profiles for the entire time horizon.

FIGURE 7 .
FIGURE 7. Objective function values comparison between centralized optimization and D-ADMM algorithm.

FIGURE 8 .
FIGURE 8.The dynamic convergence performance of the relative error in ADMM and D-ADMM (a) Independent variables; (b) objective function.

FIGURE 9 .
FIGURE 9. Temperature, load demand and solar radiation profiles within entire time horizon.

FIGURE 10 .
FIGURE 10.BES units output power with and without receding horizon control for the 15-bus system.

FIGURE 11 .
FIGURE 11.Communication network of 43-bus distribution system with eight BES units, eleven edges labeled in four colors.

FIGURE 12 .
FIGURE 12. BES output power for the 43 bus for the entire time horizon.

FIGURE 13 .
FIGURE 13.Objective function values computed by the D-ADMM algorithm and centralized optimization.
during time [50-80] minutes.The regulation converges to the new optimum with the update information.

D
. 43-BUS DISTRIBUTION SYSTEM

TABLE 2 .
Solutions under different values of ξ .