Grid Integration of Solar Photovoltaic System Using Machine Learning-Based Virtual Inertia Synthetization in Synchronverter

In recent years, the domination of power electronics-interfaced renewable energy source (RES) such as solar photovoltaic (PV) system causes grid frequency instability issue. This paper proposes a new machine learning (ML)-based virtual inertia (VI) synthetization in synchronverter topology to integrate the solar PV system and the power grid with high-frequency stability. The proposed ML-based VI is synthetized by amalgamating the action and critic network to decouple active and reactive power control. Therefore, the proposed synchronverter exhibits decoupled control and flexible moment of inertia ( $J$ ) changes that lead to high stability and fast transient response as compared to the conventional proportional-integral (PI) and fuzzy logic (FL)-based synchronverters. Various case studies in MATLAB/ Simulink simulation have been carried out, and the results proved the feasibility and effectiveness of the proposed ML-based synchronverter. Through the proposed control strategy, the maximum frequency deviation from the nominal value, settling time to reach quasi-steady-state frequency and steady-state error has been reduced by 0.1Hz, 35% and 27% respectively.


I. INTRODUCTION
Over the decades, RES such as wind turbines and PV cells, are gradually replacing conventional power generation from coal, oil or natural gas in a modern power system [1]. The gridtied RES connection via static DC-AC converters or inverters often has minimal inertial response for the stability of grid voltage and frequency due to the lack of rotating parts, unlike the synchronous machine (SM)-based conventional power generation [2]. As the penetration of distributed generation (DG) units becomes higher, the total inertia of the power system decreases over the years [3]. Hence, the frequency fluctuation of the grid cannot be damped by inertia effectively [4]. As a result, the power system under high penetration of RES is expected to become less predictable and unreliable dynamic non-linear system [5]. Recent researches have been focused on the adoption of a control scheme for inverters to compensate for the loss of inertia by mimicking the SM. This synthetization concept is called synchronverter The associate editor coordinating the review of this manuscript and approving it for publication was Giambattista Gruosso . or synchronous inverter [6]. With VI-based inverters, the gridconnected inverter can emulate the behavior of a real SG by mimicking the droop mechanism, of its rotational and damping characteristics.
The frequency-droop mechanism for active power loop (APL) and voltage-droop mechanism for reactive power loop (RPL) are introduced to regulate the real and reactive power. However, the synchronverter is mainly based on the assumption of inductive line impedance, instead of the complex impedance. As a result, strong coupling effects exist between the output real and reactive power. It adversely affects the stability and controls the flexibility of the system. Hence, it is necessary to introduce decoupling schemes to reduce the coupling effects and improve the control stability [7].
Although the VI-based synchronverter seems to be a feasible solution for the interface of the grid-connected solar PV system, it exhibits mathematical instability, strong coupling between APL and RPL, low analogue precision and unsatisfactory transient response [8]. In this paper, a full digitalized power decoupling control schemes based on ML topology for the synchronverter are proposed. It is to VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ replace the conventional PI controller. Theoretically, a gridconnected solar PV system is a non-linear system because of the characteristic of current-voltage (I-V) and power-voltage (P-V) curve in a solar array. A traditional linearized analogue feedback PI controller is commonly used to control the synchronverter based on a linear controller for the linearized system model. By varying the operating point and the system parameters, the behaviours of the controller are subject to change. Apart from PI-based controller, the non-linear controllers with feedback linearization technique are popular with replacing the linear controllers for better performance. Unfortunately, the non-linear controllers are complicated to design and they require exact system parameters. Hence, artificial intelligent (AI)-based controller is more feasible and relevant for synchronverter controls. The most notable AI controllers for synchronverter are FL [9] and adaptive neuro-fuzzy inference system (ANFIS) [10]. Apart from AI, model predictive control (MPC) is also a popular technique to control inverter [11]. Hence, a novel control strategy for synchronverter based on reinforcement learning (RL) is proposed to emulate its adaptive VI. The application of AI is crucial for the balance between performance and complexity. The difference between the existing researches and the proposed RL control strategy is the proposed strategy has the ability to identify a time-varying system through continuous learning by adapting different operating variables through time. As an attempt to address the aforementioned limitations, this paper proposes a unified ML-based designing approach of power decoupling control strategy for synchronverter featuring VI synthesis. The main contribution of this paper is to present a new method for developing an ML-based strategy combined with the VI for controlling a grid-connected synchronverter in a solar energy system. Other contributions can be summarized as follows: • A novel ML-based VI synthetization for synchronverter is proposed by introducing the neural network (NN) and ML control strategy; • Steady-state grid frequency deviation and frequency recovery delay are eliminated during the fluctuation of loads and overloading fault; • ML control is integrated into a power decoupling APL and RPL being the first of its kind, enabling the adaptive synthetic inertia (J) and damping factor (D P ); • ML controller is an optimal control designing procedure that enables the scalability and adaptability of synchronverter with guaranteed steady-state and dynamic performances; and • The proposed controller eliminates the requirement of PI gains parameter tuning efforts. The remainder of this paper is organized as follows: Section II presents the typical control scheme of PI-based and FL-based synchronverters, followed by the proposed ML control strategy implementation in Section III. Section IV shows the problem formulation and simulation setup, where the results of the simulation have been analyzed in terms of the stability with the performance results shown in Section V, and Section VI concludes the paper by suggesting future research directions.

II. CONTROL SCHEME OF A CONVENTIONAL PI-BASED AND FL-BASED SYNCHRONVERTERS
A. PI-BASED SYNCHRONVERTER Generally, a conventional PI-based synchronverter is modelled using SM's swing equations [12]. The swing equations are responsible for injecting or absorbing active power in response to frequency deviation. The PI controllers are used to ensure that the operation of synchronverter is based on the second-order swing equations [13]. Fig. 1 shows the application of synchronverter in a grid-connected solar PV system to convert solar DC source to AC power output.
The per-phase winding resistance and per-phase synchronous inductive reactance are denoted by R and X S respectively. An LC circuit consists of an inductor (L) and a capacitor (C) connected in parallel is used to filter current and voltage ripples. V g is the three-phase AC power grid, V abc and I abc are the measured three-phase voltage and current at point of common coupling (PCC) respectively, and R abc is the local resistive load. The main advantage of a conventional PI-based synchronverter is the robust and dynamic control of active power (P) and reactive power (Q) [14]. Synchronverter-based synthetization can be applied in an energy storage system (ESS) to supply power in demand [15]. The parameters of synchronverter are tuned to achieve the desired transient and steady-state response [16]. An auxiliary loop is added as a damping correction loop to adjust the dynamic response speed freely [17]. The inertial response is defined as the power exchange between the mechanical rotor and the electrical grid due to the rotor swing, which consists of acceleration or deceleration.
As illustrated in Fig. 2, the PI-based controller of synchronverter is based on both active power and reactive power droop controls where P SET and Q SET are the setpoints of active and reactive power; ω n is the nominal angular frequency; ω is the difference between measured and nominal angular frequency; T m is the mechanical torque; T e is the electromagnetic torque; T is the difference between T m and T e ; M f i f is the field excitation; 1/s is the integrator; D P is the damping factor; K Pf and K if are the PI regulator 49962 VOLUME 8, 2020 coefficient; J is moment of inertia; V g is the grid voltage; e is the control signal of pulse width modulation (PWM); θ is the angular angle; Q is reactive power and k is the coefficient for reactive power droop control. The control signal (e) output is to drive the PWM of switching devices in an inverter. The three essential swing equations (1), (2) and (3) are responsible for emulating the inertial characteristic of an SM.
where reactive power, angular speed, mutual-inductance, field excitation current, output current, electromagnetic torque and control signal are denoted by Q,θ, M f , i f , i, T e and e respectively. , marks the inner product. The cos θ and sin θ are vectors defined as the three-phase angle difference which is equally spaced in 120 • or 2π/3 in radian, as shown in equation (4).
VI is defined as a property of the grid, which limits frequency variations in the case of sudden load change or overloading fault. The rotor speed of an SM is maintained by the prime mover and the damping factor of APL (D p ) which is caused by mechanical friction. It is important to share the load evenly in proportional to the nominal load by varying the real power delivered to the grid according to the grid frequency. It is executed by a control loop, known as active power droop control. When the real power demand is higher, the speed of the synchronous generator drops because of increasing T e , governed by formula (4). where J is the moment of inertia,θ is the rate of change oḟ θ, T m is the mechanical torque andθ g is the grid angular frequency. Although the conventional PI-based synchronverter is equipped with VI based on swing equations, it is incapable of adapting to changes in operating conditions, especially in a dynamic solar PV system with varying irradiance and temperature [18].

B. FL-BASED SYNCHRONVERTER
Apart from the conventional PI control method, there are various AI-based control methods applied in a synchronverter.
As illustrated in Fig. 3, the supplementary controller of PI-based synchronverter is realized through FL control to reduce frequency deviations during disturbances. The FL inputs are typically based on the frequency deviation error (e f ) and its rate of change ( e f ). The output is the moment of virtual inertia (J ) which is adaptive based on the change of frequency error. In general, all AI-based controllers for synchronverter exhibit faster convergence speed compared to other control strategies because of its adaptive J and intelligent control of P. The most notable AI controllers are FL [19], fuzzy secondary controller (FSC) [9], fuzzy adaptive differential evolution (FADE) [20], adaptive neuro-fuzzy inference system (ANFIS) [10], adaptive dynamic programming (ADP) [18] and fuzzy adaptable logic controller based on differential evolution (DE) [20]. Apart from AI, model predictive control (MPC) is also a popular technique to control inverter [11].

C. THE MATHEMATICAL MODELLING OF A CONVENTIONAL VI-BASED SYNCHRONVERTER
This section discusses the proposed ML-based VI emulation for synchronverter by using power decoupling approach. By taking the voltage and frequency measurements at the PCC, the ML controller determines future control active power inputs by solving an optimization problem over a finite horizon. It integrates the references and predictions of future states of a synchronverter by using the past and present measurements. Fig. 4 shows the block diagram of a general synchronverter, where P t is the mechanical power; P e is the electromagnetic power; ω is the angular frequency; θ is the virtual rotor angle; E mag is the electromagnetic generated voltage;É is the generated voltage; V d_ref is the reference value of d axis voltage; V q_ref is the reference value of q-axis voltage; V r is the reference voltage;Í is the current; R is the perphase winding resistance and X s is the per-phase synchronous VOLUME 8, 2020 inductive reactance. The PI block represents the PI mathematical operation while abc-dq block represents the three-phase (abc) signal to dq rotating reference frame transformation. Sinusoidal pulse width modulation (SPWM) is a technique of PWM used in inverters to produce AC output from a DC output by using switching circuits. It is to simulate the sinusoidal wave by producing one or more square pulses of voltage (V ) per half cycle. The general structure of synchronverter controller represents the swing equations of a typical SM [21]. The equation of APL regulating mechanism is shown in equation (6) while the equation of RPL regulating mechanism is shown in equation (7).
where K is the regulator coefficient of reactive power, D q is the damping factor of RPL,θ r is the nominal angular frequency,θ is the real-time angular frequency, Q SET is the set point of reactive power, V r is the reference voltage and V m is the measured feedback voltage. By selecting the synchronous rotation axis as the reference axis and based on the relationship between the electrical angular velocity and the mechanical angular velocity, we derive the rotor equation of motion (8).
where ω, ω n , P T and P e are defined as electrical angular velocity, synchronous electrical angular velocity, mechanical power and electromagnetic power respectively. The voltages at PCC are balanced when θ b = θ a − 2π/3 =θ c +2π/3 and hence measured voltage amplitude (V m ) is computed by the equation (9).
where V a , V b and V c are defined as three phases voltage. The behavior of the inverter is dependent on the parameters of inertia and damping properties (J and D p ) from the grid stability point of view. The adjustable parameters of synchronverter are able to further optimize the response of inverter to utility grid behavior, as compared to synchronous generators. For example, during the occurrence of grid ground fault, high inertia (J ) is adopted to avoid overshoots while low inertia is configured for quick recovery. The frequency droop coefficient (D p ), is calculated by equation (10) as a relation of the added active power per frequency change. In a similar way, the voltage droop coefficient (D q ) is calculated by equation (11) as added reactive power per grid voltage change.
The droop coefficients are dependent on the choice of a designer depending on specific grid codes in a certain area. The D p is usually configured such that a 0.5% drop in the grid frequency would double the active power. On the contrary, the D q is configured to double reactive power production at a 2% voltage drop [12].

III. NOVEL VIRTUAL INERTIA CONTROL ALGORITHM BASED ON MACHINE LEARNING TOPOLOGY WITH DECOUPLED POWER A. PROPOSED MACHINE LEARNING TOPOLOGY WITH POWER DECOUPLING ABILITY
Inspired by the learning process of living organisms, RL is a simple iterative algorithm which acts on its environment, observes the resulting reward and manipulates its actions accordingly to maximize the receiving reward. The actionbased RL, which is also the latest frontier of ML, is able to capture ideas of optimal behaviour occurring in a dynamic system. The mathematical formulations of RL and its practical implementation method is known as ADP. ADP is a type of online ML optimal control that approximates an optimal control scheme. It learns by trial and error to achieve an objective.
As illustrated in Fig. 5, an ML controller based on the topology of RL has been proposed for the optimization of a synchronverter. The RL algorithm explores the system of synchronverter to gain experience. It learns to inject or absorb active power (P) optimally in response to the frequency tracking error ( f ). The design of RL-based controller is intended for man-made engineered systems like synchronverter to equip it with the capabilities to learn and exhibit optimal behaviour. The pseudo-code is shown in Algorithm 1 as followed.

Algorithm 1 Computational Adaptive Optimal Control
Inputs: Grid operating frequency (f g ) and nominal frequency (f n = 50Hz) Outputs: Active power (P) Based on: ω-P droop relationship Initialization: 1. Initialize the Q-values table, Q (s, a) and immediate rewards r (s, a) = 0.0, in such ∀s ∈ S,∀a ∈ A 2. Repeat for a given number of operating points over the whole planning period: Start a) Observe the current grid frequency state vector (s); b) Choose an action vector (a) to increase or decrease P, for that state based on action selection policy; c) Take action by injecting or absorbing P and observe the reward, r and the new grid frequency state, s ; d) Update the Q-value for the state using the observed reward and the maximum reward possible for the next state; e) Set the state to the new state, and repeat the process until a terminal state -quasi-steady-state frequency i.e. f is almost 0 is reached; Until Q-function converges to Q * .
The objective of the ML-based controller is to minimize the f which results in lower frequency oscillation, overshooting and steady-state error, thereby stabilizing the frequency response of synchronverter AC output. It is to ensure the value of f g is as close as the f n . The RL-based controller is then rewarded or penalized depending on whether its behavior (injection or absorption of P) helps or prevents it from reaching its objective ( f = 0). The algorithm updates the q-matrix values depending on the reward value. The discount factor (γ ) of the return computation has been configured to 0.95, while the learning rate (α) are configured as 0.9. The learning algorithm searches for the optimum value for the q-matrix to reach optimum control. The algorithm updates the q-matrix values depending on the reward value. In order to improve the transient response, steady-state performance and robustness of a conventional synchronverter, the ML-based controller is proposed to replace the existing PI controller. A PI-based synchronverter is characterized by three basic elements based on the RL topology: 1. Observation of states (s): The instantaneous measurement of f g is executed to compare against f n to acquire f by subtracting f g from f n , which is to be observed by the actor. 2. Action (a): The actor takes action by manipulating the synchronverter output current magnitude to inject or absorb active power accordingly. 3. Reward (r): The positive reward (+100) is given when the f is less than or equal to 0.1Hz. On the contrary, when the f is larger than 0.1Hz, negative reward (-100) is given to penalize the actor. In general, an ML-based controller exhibits self-learning capability, adaptive control and digitalized robustness compared to the PI controller. AI is a common process for computers to solve specific problems by emulating the human's behavior. ML is a subset of AI by learning from observations and make decisions based on trained models. Among the ML technologies such as supervised learning, unsupervised learning, semi-supervised learning and RL, the RL is adopted in resolving the stability issue of a synchronverter. This RL-based topology refers to an agent that learns its action policy (A) that maximizes the expected rewards (R) based on its interactions with environments.
In general, an RL agent continuously interacts with an environment; where the environment receives an action, emits new states and calculates a reward; and the agent observes states, suggests action to maximize next reward. Training an RL agent involves dynamically updating a policy (mapping from states to action), a value function (mapping from action to reward) and a model (for representing the environment). Deep learning (DL) provides a general framework for representation learning that consists of many layers of nonlinear functions mapping inputs to outputs. Its uniqueness rests with the fact that DL does not need to specify features beforehand. One typical example is deep NN [12]. Basically, DRL is a combination of DL and RL, where DL is used for representation learning and RL for decision making. In the proposed Grid Mind framework, deep Q network (DQN) [15] is used to estimate the value function, which supports continuous state sets and is suitable for power grid control.
A deep deterministic policy gradient (DDPG) agent is trained to control a second-order swing equation from synchronverter dynamic system modelled in MATLAB R Simulink. The training goal of the agent is to control the grid frequency in a second-order system by injecting or absorbing active power from solar energy. For this environment, the grid frequency starts at the initial position of its nominal value +/− 50Hz. The active power action signal from the agent to the environment is from +10W to −10W. The observations from the environment are the grid frequency (f g ) and active power at the point of common coupling (P PCC ). The episode terminates if the grid frequency is more than 51Hz or less than 49Hz from its nominal value. The reward (r t ), provided at every time step, is a discretization of r(t): where x is the state vector of the mass, u is the force applied to the mass, Q is the weights on the control performance and R is the weight on the control effort whereby R = 0.01. There are various applications of AI in synchronverter, such as ADP approach, action-critic network and online NN-based controller. The controller is a supplementary controller in addition to the conventional PI controller. The power references generated by the main synchronverter algorithm with a supplementary signal P ADP to give the total reference VOLUME 8, 2020   P TOTAL [18]. An NN-based adaptive critic design is utilized in a typical PI-based synchronverter controller to optimize its cost function. The characteristic of the NN improves the performance under uncertainties and unknown parameters such as varying irradiance and temperature [22].

IV. PROBLEM FORMULATION AND METHODOLOGY A. SYSTEM SETUP AND CONFIGURATION
In order to demonstrate the effectiveness of the proposed control scheme, a small-scale grid-connected solar PV system is modelled by using MathWorks R MATLAB/ Simulink platform with Simscape Electrical TM . Fig. 6 shows the detailed synchronverter controller without the phase lock loop (PLL).
The system topology is mainly based on the swing equations of a conventional SM and power droop control with modified power decoupling control. The performances of the proposed ML-based synchronverter have been assessed by simulation. The simulation results compare the performance of the proposed ML with a traditional PI-based and FL-based design.
The detailed synchronverter and MPPT controller are illustrated in Fig. 7 and Fig. 8 respectively. The detailed simulation model is shown in Fig. 9. The MPPT is responsible for maximum power extraction at solar DC output while the synchronverter provides a self-synchronizing DC-AC interface between solar to the power grid with stabilizing frequency output. The cascading DC-DC and DC-AC converters are necessary to interface the solar PV system with the power grid.
The simulation test system is a detailed model of a 100-kW SunPower modules (SPR-305E-WHT-D) array connected to a 25-kV step down to 400V grid via a DC-DC boost converter with IC MPPT algorithm and a three-phase two-level inverter with the synchronverter algorithm. The case studies are executed by manipulating variables which consist of solar irradiance and temperature of the solar array, synchronverter algorithm, grid or standalone connection and circuit breaker for sudden step load increase or decrease. The control strategy was simulated by the solver ode23tb with a maximum step size of 0.5ms. The synchronverter consists of two major parts, namely the power part and the electronic part. The power part is essentially the power circuit of inverter or DC-AC converter and the electronic part is the controller of synchronverter [12]. It is used to manipulate the PWM to drive the gate of a switching device in an inverter [23]. The mathematical modelling of synchronverter enables the inverter to behave like an SM which is self-synchronizing with the grid frequency. The 2-level bridge of insulated-gate bipolar transistor (IGBT) gate inputs are controlled by the PWM of synchronverter intelligent control. An LC filter filters the ripple voltage and current by using the capacitor and inductor respectively. The resistive load at DC (R DC ) output of DC-DC boost converter is used to emulate the local resistive load for the DC side of the solar PV system. It is also known as a resistance load or bleeder resistance that is connected across the output such that even though no external load is connected, the converter sees a minimum load for stable operation.
To verify the effectiveness of the proposed ML-based controller, the simulation model of a synchronverter was built using the parameters as shown in Table 1. The 100-kW solar PV array is configured using the parameters listed in Table 2 for simulation. The maximum power point tracking (MPPT) is implemented in the boost DC-DC converter by using the IC with an integral regulator technique. Different schemes of synchronverter controller which includes conventional PI, FL and ML are designed to interface grid-connected PV system under different temperature and irradiation. The operating point of the PV on the I-V curve is dynamically modified by the DC-DC controller so that the MPPT obtained the maximum power point (MPP) at any sunlight conditions and maintained PV power in the proximity of this point to produce power with the higher efficiency. The control systems of synchronverter are studied, with the conventional PI in Fig. 7, FL controller in active power droop on top of the conventional PI control in Fig. 10, and ML-based controller is then proposed in Fig. 11. The MPPT is not contributed to the inertial synthetization due to its DC-DC conversion. The inverter is the key element for inertial synthetization due to   its ability to inject or absorb active power by the conversion process from DC to AC.
An FL controller fetches the inputs of frequency deviation error (e f ) and its rate of change of error ( e f ). It converts them into fuzzy membership function and decides the value of the moment of inertia (J ). For large error, the J will be high to dampen the error or vice versa. The case studies are designed to compare, analyze and evaluate each performance with the integration of synchronverter. All other blocks' parameters are retained as a constant environment to ensure that only synchronverter controller is an only varying factor. In general,  it consists of a conventional inverter with a synchronverter control which interfaces solar PV DC output to AC grid. The three-phase two-level inverter generates an AC output voltage waveform that can be controlled by PWM. The PWM is generated by the synchronverter controller. The three-phase inverter uses a DC power supply which emulates the solar PV output. The gate driver signals to produce balanced three-phase sinusoidal output which is connected to the grid.  With a three-phase voltage and current measurement block as illustrated below, the synchronverter can be controlled by using feedback measurement through voltage and current sensor. The accelerated solver is ode45 with variable-step. The maximum and minimum step sizes are set as adaptive automatic and the relative tolerance is set as 0.001. The mode of operation is unsynchronized with 10kHz of switching frequency. Its initial phase is at 90 degrees with the natural sampling technique. Its sample time is configured as 1e-6 second. The imaginary mechanical friction coefficient and frequency drooping coefficient (D P ) is set as 0.2. This value indicates that if there is a drop of 0.5% in the frequency from the nominal frequency, it causes the torque to increase by 100% of the nominal power. In contrast, the voltage droop coefficient is chosen as D q = 117.88 resulting in a drop of 5% in the voltage causes an increase of 100% reactive power. The current ripple is limited to +−5%, while the voltage drop on the inductor is limited to less than 10%.
The system model used in this study consists of a fully detailed switching voltage source converter (VSC) model, whose parameters are set out in Table 1 and Table 2 respectively. A detailed synchronverter-based controller was built in MATLAB/Simulink environment. The reason for building a fully detailed system is to study and compare steadystate and dynamic properties during standalone and gridconnected operation, which occurs with sudden step load increase or decrease and instantaneous short circuits fault incident. The existing FL and conventional PI control approaches proposed in [9] and [12] respectively were considered. The study presented here was concentrated on four important factors: the amount of active power (P), the generation of VI, the frequencies of system, and the value of virtual damping. The comparison of the two VSM algorithms was based on their dynamic properties, the total voltage and the current harmonic distortion, sudden load changes, and unbalanced AC voltages. Moreover, the switching control techniques for both high-order and low-order VSM are based on PWM switching concept. All simulations and case studies are executed under similar simulation environment configuration   with similar settings and values to ensure the consistency and accuracy of the result. The only variable is the replacement of controller which consists of conventional PI, FL and proposed ML-based control strategy.
A single line diagram of the grid-connected solar PV system model is a built-in MATLAB/ Simulink simulation. The system model used in this study consists of an average VSC model. As shown in Table 3, the simulation was conducted in six scenarios to evaluate the performance and robustness of the proposed ML-based controller compared to conventional PI and FL-based controller for synchronverter: case studies 1 -6. The PI controller is most commonly used in the industry and has been universally accepted in industrial control because of its robustness and functional simplicity. Thus, to design the ML controller, a PI controller was computed and executed first. This case investigates the difference between conventional PI [19], FL [24] and ML-based synchronverter controller under different case studies. The case studies demonstrate the influence of MPPT tracking operation and varying irradiance and temperature of the solar panel. The constant irradiance and temperature of the solar panel are standardized as 1000W/m 2 and 50 • C for its peak value. However, for case studies 5 and 6, the irradiance and temperature have been varied to emulate real-world operating environment as shown in Fig. 12 and Table 4.   Simulations for different mode and resistive load are conducted as the resistive load is the first stage of research. At the beginning of the simulation process, only load 1 and 2 are connected to the synchronverter output. For load increase case, at 3 rd second, load 3 is turned on and connected to the system via the circuit breaker. For the load decrease case, initially, the load 3 is connected and at 3 rd second, load 3 is turned off by the tripping of the circuit breaker. Table 5 presents the standardized operating condition for all case studies.

V. THE ANALYSIS OF SIMULATIONS A. FREQUENCY OUTPUT ANALYSIS UNDER OVERLOADING FAULT
In the fault analysis, an overloading scenario caused by the tripping of generators and transmission lines is created to emulate a sudden fault occurrence at 0.1 seconds. It is to investigate the frequency response of synchronverter under conventional PI, FL and proposed ML control schemes. The frequency profiles at the PCC between synchronverter output and the cascaded connection of residential power grid (230V RMS ) as well as local resistive load are presented in Fig. 13 (a)-(f) for case studies 1-6 respectively.
The first case study aims at verifying the transient performance of the proposed ML-based synchronverter in term of frequency recovery speed. Fig. 13 (a)-(f) presents the transient behaviors of the synchronverter AC output in response to overloading fault, which is occurred at 0.1 st second. The overloading fault is emulated by the condition of sudden load connection or disconnection. It can be observed that the power system frequency declines due to the imbalance of power generation and consumption. The frequency fluctuation during the occurrence of overloading fault is found to be less using the proposed ML scheme than when using conventional PI and FL. Therefore, the proposed scheme is a better performer to recover frequency drop under overloading fault. It shows superior recovery performance with faster transient recovery time, higher frequency nadir and smaller overshoot.

B. FREQUENCY OUTPUT ANALYSIS UNDER SUDDEN LOAD CHANGE
The performance of the proposed ML-based synchronverter is further verified in case study B. For sudden load change analysis, an abrupt load change is applied to the local resistive load (R LOAD ) by increasing or decreasing 1.2kW (80 ). A circuit breaker (CB) is used to connect additional resistive load or disconnect the resistive load to emulate sudden load increase and decrease respectively. The proposed ML-based and FL-based synchronverter exhibit minimal disturbance in terms of their frequency output under sudden load changes. As shown in Fig. 14 (a)-(f), the frequency of conventional PI-based synchronverter is influenced by sudden load change, i.e. when the load increases, the frequency decreases or vice versa, due to the imbalance of power demand and generation. The frequency fluctuation during the occurrence of sudden load variation is found to be minimal using the proposed ML scheme than when using the conventional PI and FL. The simulation indicates that irradiance and temperature variation do not affect the frequency output of ML-based synchronverter significantly.

C. FREQUENCY TRACKING ERROR
In this case study, the frequency tracking errors among conventional PI, FL and the proposed ML-based synchronverter are compared. The frequency tracking error (f t_error ) is defined as the difference between synchronverter frequency output (f out ) and grid frequency output (f g ) that it is meant to mimic. This error implies how good the synchronverter performs to keep track of the grid frequency and follow its nominal operating frequency. As shown in Fig. 15 (a)-(f), all case studies demonstrate the optimal performance of ML-based synchronverter. It exhibits minimal frequency tracking error even when it is under overloading fault, sudden 49970 VOLUME 8, 2020 load change as well as varying irradiance and temperature. It also recovers faster from high tracking error during fault occurrence at 0.1 seconds with lower overshoot, less frequency tracking error, less steady-state error and shorter transient time.

D. ACTIVE POWER REACTION BY SYNCHRONVERTER
The active power reaction by synchronverter is analyzed to understand their dynamic behaviour in response to the frequency variation. It is based on the synthetization of a conventional SM whereby it will inject or absorb active power in corresponding to the decrease or increase of operating frequency respectively. In this analysis, Fig. 16 (a) -(f) presents the injection or absorption of active power (P) by all three types of synchronverters in response to various frequency deviation. Theoretically, a conventional SM injects active power (P) when the frequency drops because of increased power demand, or vice versa. Hence, to mimic the dynamic characteristic of an SM, synchronverter is required to inject or absorb active power when required during frequency fluctuation. Based on all the case studies demonstrated, it is found that the proposed ML control scheme requires lesser active power to bring back the system frequency to its nominal value. The reduction of required active power leads to higher energy efficiency due to lower energy consumption. The proposed synchronverter is able to retain a higher amount of active power for the resistive load with a desirable power factor (p.f.).
From all four analyses, the ML-based synchronverter exhibits good performance in self-synchronizing with the changing power grid frequency. It validates that whenever there is sudden load change, the addition of other sources with different frequency, grid instability or fault, the synchronverter is able to maintain the steady output with standardized VOLUME 8, 2020 expected frequency. In short, it provides stability, efficiency and higher power quality in interfacing solar PV to the power grid.

E. STUDY ON THE EFFECT OF VARYING MOMENT OF INERTIA (J)
The advantages of the proposed ML approach have been compared to the PI and FL approaches, based on the similar condition in simulation. In particular, the proposed method with the characteristics of decoupled control can accelerate the startup operational speed and avoid current surge at startup or in parallel operation. If there is any faulty system, the proposed synchronverter is able to establish short-time voltage support for the important load in the isolated network. It translates into the reduction of the load blackout time, and thus improves the reliability of the grid-connected RES. Statistically, the transient time, frequency recovery rate and steady-state error of the proposed control strategy have been greatly improved compared to the conventional PI controller.
In this analysis, the moment of inertia (J ) parameter is varied throughout the simulations to investigate its effect on the frequency response and active power output of the proposed synchronverter. As shown in Fig. 17, the grid frequency responses are shown with different J values. It shows that the higher the value of J , the frequency response is slower, underdamped and smoother without any visible oscillation. Meanwhile, when the value of J is lower, the frequency response is faster but accompanied with overshoot, steadystate error and oscillation. The transient responses of active power droop control when J increases from 0.2 to 0.3, and then from 0.3 to 0.6, are investigated to study the effect of J on active power output response. The value of J is varied between 0.5 to 1.5 because J is equivalent to the time constant of the frequency loop multiplied with the damping factor (D P ). It is manipulated to study its influence on the effectiveness of frequency tracking.
To study the effect of J , at t = 0.5s, the grid frequency drops from its nominal frequency 50Hz to 49.8Hz by  0.2Hz or 0.4%, which is caused by a sudden load increase. At t = 1.0s, sudden load decrease has occurred and it caused the grid frequency increases by 0.2Hz or 0.4% from its nominal frequency, 50Hz to 50.2Hz. The corresponding active power of synchronverter is shown in Fig. 18. At t = 0.5s, the active power (P) increase from its nominal value (P SET ) 10kW to 12kW by 2kW or 20% to recover the grid frequency. It compensates the required active power in response of frequency drop by imitating the inertial response of an SM. The VI synthetization mimics the speeding up of the rotational movement of an SM by injecting more active power when there is frequency drop due to sudden load change. The injection of active power is done to recover the frequency back to its nominal value, thereby stabilizing the grid frequency and retaining power quality. Similarly, at t = 1.0s, 2kW active power has been absorbed by the synchronverter with the increased 0.2Hz grid frequency. Both injection and absorption of active power obey the drooping characteristic of 2% change in frequency ( f ) over 100% change in active power ( P). In this case, the frequency increases or decreases by (0.2Hz) 0.4%, which results in (2kW) 20% of active power absorption or injection respectively. The transition times for J = 0.2, 0.3 and 0.6 are approximately 0.1s, 0.2s and 0.4s respectively. It implies that the higher value of J increases the transition time but decreases unwanted oscillation and steady-state error due to its dampening factor.
In this analysis, the stability of the proposed synchronverter is evaluated by the manipulation of moment of inertia (J ) to observe the frequency response of the system. From Fig. 19 and 20, it is observed that when the J is increased, the dampening effect is obvious to stabilize the frequency oscillation caused by the faults and overloading. However,   the balance between fast response and the dampening effect is crucial to minimize overshot or under damp.
As illustrated in Fig. 21, the result shows that the proposed synchronverter remains stable when J = 0.6 even when under the high penetration of solar energy with the power grid. It exhibits high stability control because of the balance between frequency response time and damping effect with the desirable J value. The higher the J value, the faster its transient response but with higher overshoot and oscillation (steady-state error). The balance between fast transient response and low steady-state error is crucial, hence the choice of J value should be appropriate for its balance.   overloading fault condition. The overloading fault is simulated by increasing resistive load to exceed the rated load. As shown in Fig. 22, at t = 0.1s, the overloading fault has occurred and hence the grid frequency (f g ) drops from its nominal value (f n ), 50Hz. The value of H is varied by 4 seconds because it is calculated by the ratio of the kinetic energy (synchronous inertia) and the nominal active power of the machine. The H of a standard SG is 4s, hence the H values at 1s and 8s show the implication of deviated H from its standard value. It is observed that the higher the value of H , the lower the frequency drop (f error ). This is due to the higher value of H resulting in higher kinetic energy stored at the synchronous speed. Hence, there are more available active power to compensate for the frequency drop. From this finding, the higher value of H corresponds to an SM which is able to change speed over long time periods although it responds slowly to transient mismatch of power input to power output. Moreover, the lower value of H corresponds to an SM which is able to change speed faster. Hence, high H exhibits high stability, high dampening effect, less overshooting and oscillation, although it takes longer to recover and settle down. In the case of the synchronverter, the speed to change quickly is not required, therefore higher H is preferable for better stability and recovery.

G. TABULAR RESULTS EVALUATION
This section presents the summary findings on six case studies of different synchronverter control schemes. As shown in Table 6 and Fig. 23, it can be easily observed that ML-based synchronverter performs better compared to conventional PI and FL-based synchronverter in all case studies.
Generally, the proposed ML-based synchronverter performs better because of its adaptive learning behaviour, digitalized control and adjustable damping factor (D P ) and inertia constant (J ). From the tabular results, it is observed that the proposed ML-based synchronverter has higher frequency nadir, lower frequency drop or deviation, lesser overshoot, lower standard deviation, smaller steady-state error, faster settling time and lesser injected or absorbed active power (P) with high energy efficiency. Despite the frequency overshoot and absorbed active power of FL-based synchronverter which are desirable than ML-based controller in some cases, the difference is minimal and negligible. The computational time of ML-based synchronverter is expected to be approximately 1.5 times more complex than the conventional synchronverter but is better than FL-based synchronverter, which exhibits two times more complex than the conventional method. The computational burden of the ML controller is reduced. Hence, its structure is simpler and improves the response time of the controller. It is found that when the synchronverter is controlled using the proposed ML control scheme, the frequency deviation from its nominal value is very small and it takes the least time for the frequency to return to its nominal values after the fault is cleared. However, for a similar event, the conventional PI and FL lead to a greater frequency deviation and a longer time to return to their nominal values. The cost-effectiveness, algorithm complexity and computational requirement of virtual inertia synthetization should be considered for performance benchmarking. Generally, the ML-based control system is recommended for the solution of electric power system decision or control problems to overcome the highly complex power system. Overall, the simulation results validated that the ML algorithm is an effective control method for the design of synchronverter based on the following reasons.
1. ML method does not make any strong assumptions on the synchronverter system and it is able to cope with partial information on non-linear behaviour. 2. It opens avenues to adaptive control since the agent learns continuously and can adapt to changing operating conditions, especially in the dynamic solar PV system, without the need for manual retuning. 3. ML can be used in combination with traditional PI control method to balance the performance and stability of synchronverter.

VI. CONCLUSION
In this paper, a novel ML-based VI synthetization control strategy for the grid integration of solar PV system based on the synchronverter is proposed. A typical synchronverter with ML-based controller is designed to integrate solar PV system with power grid while maintaining high-frequency stability even when it is under varying irradiance and temperature. The simulation results from MATLAB/ Simulink verified the promising transient frequency recovery speed and steady-state performance (frequency oscillation) of the proposed strategy, even when under overloading fault condition and sudden load changes. The future works include the application of the proposed ML-based VI synthetization with power decoupling feature for static synchronous compensator (STATCOM), flexible alternating current transmission system (FACTS), high voltage direct current (HVDC) transmission and other RES grid interfacing system.