Faster-Than-Real-Time Hardware Emulation of Transients and Dynamics of a Grid of Microgrids

Enhanced environmental standards are leading to an increasing proportion of microgrids (MGs) being integrated with renewable energy resources in modern power systems, which brings new challenges to simulate such a complex system. In this work, comprehensive modeling of a grid of microgrids for faster-than-real-time (FTRT) emulation is proposed, which can be utilized in the energy control center for contingencies analysis and dynamic security assessment. Electromagnetic transient (EMT) modeling is applied to the microgrid in order to reflect the detailed device processes of the converter and renewable energy sources, while the AC grid utilizes the transient stability modeling to reduce the computational burden and obtain a high acceleration value over real-time execution. Consequently, a dynamic power injection interface is proposed for the coexistence of the two simulation types. The reconfigurability and parallelism of the field-programmable gate arrays (FPGAs) enable the whole system to be executed in FTRT mode with 51 times acceleration over real-time. Meanwhile, three case studies are emulated and the results are validated by the off-line simulation tool Matlab/Simulink®.


I. INTRODUCTION
T HE penetration of microgrids (MGs) with renewable energy resources has been increasing in the power system to alleviate the energy crisis and environmental issues [1]. The traditional centralized structure of the power systems is not efficient to meet the growing electricity demand due to the power loss in the transmission network [2]. Therefore, modern power systems are experiencing a shift from centralized generation to distributed generation [3]. The distributed generation with multiple MGs integrated may bring new challenges including designing, operating, and coordinating the complex system. Considering that in such an integrated network a small disturbance may spread to other areas and cause severe damage to the whole system, it further increases the complexity of controlling the microgrid cluster.
To deal with the complexity caused by the integration of MGs, a variety of models and control strategies have been developed and investigated in the literature. Most of the existing microgrid technologies focus on one specific aspect, such as energy management, control methods [4], [5], [6], [7], optimal power flow [8], or protection schemes [9], [10]. Meanwhile, some simulation models for analyzing the microgrids are also investigated, e.g., the detailed and simplified models for the energy storage system (ESS) in MGs are presented in [11] and [12]. Although significant progress has been made, these research results lack hardware support and the simulation models mentioned above are hard to realize in real-time, which is insufficient for modern energy control center that requires taking remedial actions immediately after a disturbance. Furthermore, it falls short of revealing the impact of multiple MGs on the AC grid with which they are integrated.
While there is still further research needed related to real-time simulation of the MGs, significant progress has been made in the literature which can be categorized into four aspects: 1) the design and modeling of MG components to achieve real-time simulation [13], [14], [15], [16], [17], [18], [19], [20]; 2) the utilization of existing commercial real-time simulators [21], [22], [23], [24]; 3) the development of real-time virtual test bed for MGs [25], [26], [27]; 4) novel computational approaches for distribution grids [29], [30]. Real-time simulation prefers the models with lower computational burden, and therefore, the relatively simpler models or equivalent dynamic models have been developed. Machine learning (ML) based models have also been investigated to selectively model and simulate MG components [15], [16], which significantly reduced the hardware consumption; however, the ML-based models may ignore some dynamics of the MG components to obtain a higher simulation speed. Thus, these modeling approaches still require further research to meet the requirements of modern power systems for dynamic security assessment. Commercial real-time simulators are usually executed on high-performance processors or supercomputers to realize the high-speed simulation for microgrids [35], which are able to conduct the simulation of MG components in real-time. Although their accuracy and efficiency can be guaranteed due to the high processing frequency, the cost of the computing equipment is the main factor that limits their widespread application. Meanwhile, the hardware-in-the-loop (HIL) simulation is also utilized in modern power systems and has also been applied to investigating microgrids [26], [36]. The hardware resources of the existing simulators restricts the scale of MG or MG clusters. Only a relatively small scale microgrid is tested and neither power dispatch nor interactions among grid components are investigated in the above papers. Virtual testbeds for cyber and physical data acquisition have caught a lot of attention in MG simulation, which can also coordinate with other unconventional modeling methods to realize real-time simulation [27]. Due to their flexibility and scalability, virtual testbeds are convenient to reconfigure and adapt to various systems without depending on specialized hardware; however, they have limited real-time simulation capabilities, especially in applications that require deterministic, low-latency feedback loops for real hardware [27]. The lack of efficient physical interfaces is also another one of their drawbacks. Furthermore, some novel approaches were also proposed for real-time simulation of MGs. Commonly, EMT simulation method is adopted for detailed representation of MG components described by non-linear differential algebraic equations. Parallel solution approaches were therefore put forward to gain a high-speed execution. These methods usually require sufficient hardware resources for parallel processing, and multiple FPGA boards are utilized to realize real-time simulation [30], [31].
Recently, the FPGA-based platform has caught a lot of attention due to its strong computation capability. The FPGA is capable of performing high-speed real-time simulations with time-steps in the sub-microsecond range [32], [33], [29], however, a single FPGA board is limited by its finite amount of logical resources to perform parallel computations in solving a large-scale system. As presented in [34], a large system is deployed on a single FPGA board by utilizing parallel computation method, and the real-time simulation can be achieved. However, the steady-state estimation is taken into consideration in [34], which only provides a single state power estimation. The emulation of modern power grids requires each circuit component to be modeled in detail to properly reflect the dynamic processed and performance of the system. Nevertheless, low latency and hardware resource occupation are also critical for industrial practice. In this work, comprehensive modeling of microgrids for faster-thanreal-time (FTRT) emulation on multiple FPGA boards is proposed, which can not only provide real-time HIL simulation services for testing local MG control and protection functions, but also enable the energy control center with effective strategies to improve the stability and security of the larger grid. It is quite challenging, even for real-time HIL simulation, to model and emulate a microgrid cluster.
Compared with the available commercial real-time simulation tools, the FPGA-based FTRT emulation has following advantages: firstly, both the FPGAs and RT tools are able to realize the HIL emulation, however, the FPGAs can achieve the FTRT emulation with the help of a creative solution algorithm, and efficient parallel implementation, which is much faster than the RT tools. Secondly, the capability and scalability of the FPGA-based FTRT platform are better than currently available commercial RT simulators. Take the case study in [32] (which was done with RTDS Technologies Inc., Winnipeg, Canada) for example, the 141-bus system with 38 generators is simulated using RTDS and 4 PB5 racks were needed [32]. To reduce the hardware resource cost, only 5 buses and 2 generators are simulated on RTDS, while the rest of the system parts are simulated on FPGA boards. The cost and the hardware resource occupation of an FPGA-based platform are much lower than the RT simulators if the system scale becomes larger. Due to its accelerated mode of execution, an FTRT emulator can conduct traditional control center functions such as dynamic state estimation, power flow, and contingency analysis much faster to predict the system condition in response to adverse events. It can then run multiple scenarios in parallel to devise and recommend viable solutions to dynamically restore the voltages and frequencies to nominal values.
For a realistic power system, the role of the FTRT emulation might be more prominent, as it is able to collect real-time data from the field and provide an optimal solution without cutting off the fault area. FTRT emulation improves the grid stability by predicting the grid performance, which is beyond the capability of real-time (RT) simulation tools [37]. Furthermore, the FPGAs require lower cost and power consumption compared with RT tools, enabling them to be of service in dynamic security assessment (DSA). With the help of creative solution algorithm and efficient parallel implementation, FTRT emulation provides sufficient time for DSA to take remedial actions [38].
Apart from the hardware, the solution strategy also has a significant impact on emulation efficiency. To reflect the detailed operating conditions of the components in MGs, the electromagnetic transient (EMT) simulation is adopted, while transient stability simulation is applied to the host system to reduce the hardware resource utilization. Due to the various simulation types, a dynamic voltage injection interface strategy for hardware emulation with less data communication is also proposed, which significantly reduces the hardware resource utilization and accelerates the hardware emulation.
The rest of this paper is expanded as follows: Section II introduces the detailed modeling of the components in a typical microgrid. The interface strategy of the integrated AC system and microgrids is specified in Section III. Section IV demonstrates the hardware design of proposed FTRT emulation on FPGAs. The validation of EMT-dynamic emulation results of MG cluster and case studies for power dispatch is given in Section V. Section VI presents the conclusion and future work. Fig. 1 (a) provides the equivalent circuit representation of a solar cell, which consists of an irradiance-dependent current source, an anti-parallel diode, shunt resistor (R p ), and series resistor (R s ) [39]. The output current of the single solar cell can be expressed based on Kirchhoff's Current Law as:

II. DETAILED EMT MODELING OF MICROGRID COMPONENTS A. PHOTOVOLTAIC (PV) AND BATTERY ENERGY STORAGE SYSTEM (BESS) EMT MODEL
where i irr and i dio refer to the irradiance current and the current flowing through the anti-parallel diode as given in (2) and (3), respectively.
where the variables with the subscription ref are the reference values, G denotes the solar irradiance, α T refers to the temperature coefficient, V T is the thermal voltage, and T represents the absolute temperature. Meanwhile, I 0 is the diode saturation current. In a practical PV array, a large amount of PV panels are arranged in an array to provide sufficient energy. The topology of a typical PV array with N p parallel strings and each of them containing N s series panels is given in Fig. 1 (c). The equivalent circuit is still available in a PV array, where the (1) can be expanded as The non-linear nature of the anti-parallel diode makes the emulation of the solar cell complex. To reduce the computational burden as well as shrink the emulation time. The equivalent circuit in Fig. 1 (c) can be further simplified by Norton's Theorem, resulting in a two-node circuit as given in Fig. 1 where G p refers to the conductance of the parallel resistor, the G dio and I Deq are given as follows: The operation of a microgrid under the islanded mode requires energy storage system to balance the generation and demand as well as regulate the grid voltage. A battery energy storage system (BESS) is applied in each microgrid, which contains a battery system, and a DC/AC converter. To emulate the non-linear part of the battery system, the EMT simulation is applied for calculating BESS. The battery is modeled [11] as an ideal controllable voltage source in series with an equivalent internal resistance R batt , as given in Fig. 2 (a). The open-circuit voltage of the battery V oc can be represented based on the actual battery charge (it) by a non-linear equation expressed as follows.
where V 0 refers to the battery constant voltage, K is the polarisation voltage, Q represents the battery capacity, and A and B denote exponential zone amplitude and exponential zone time constant inverse, respectively. According to Kirchhoff's laws, the battery voltage (V batt ) can be derived as:  The controllers of the PV system and BESS also share some similarities since they are both based on the d-q frame, as given in Fig. 2 (b). For a PV converter, the controller regulates the DC voltage on the d-axis according to the reference voltage which is generated by the maximum power point tracking (MPPT) algorithm, while in BESS, the DC voltage or the active power is the control target. On the q-axis, depending on the grid condition, the converters can control either the PCC voltage or its reactive power.

B. EMT MODEL OF WIND TURBINE
A typical doubly-fed induction generator (DFIG) is applied in the microgrid as shown in Fig. 3, which includes an induction machine, a grid side voltage source converter (GSVSC), and a rotor side voltage source converter (RSVSC). The principle of the DFIG is that the rotor windings are fed with a backto-back voltage source converter which controls the rotor and grid currents, while the stator windings are connected to the grid. The converter controls the rotor currents, which is possible to adjust the active and reactive power fed to the gird under various wind speed and grid conditions.
For FTRT hardware emulation, the induction machine in the DFIG is represented by 5 th order differential algebraic equations (DAEs). The state-space equation of the induction machine can be expressed as: where x, y, and u are vectors that refer to the fluxes, currents, and input voltages, respectively. The DAEs of the induction machine contains 4 rotor and stator circuit equations in the α-β frame [40], given as: where the λ αs , λ βs , λ αr , and λ βr refer to the fluxes of stator and rotor in α-β frame, respectively, V αs and V βs are the input voltages, R s and R r are the stator and rotor resistance, and L s , L r , and L m represent the stator, rotor, and magnetizing inductance, respectively. The 5th differential equation which describes the mechanical dynamics is given aṡ where P and J are constant values which refer to the poles and inertia of the induction machine, ω r refers to the electrical angular velocity. The electromagnetic torque T e and mechanical torque T m can be obtained by where ρ represents the air density, r T is the wind turbine radius, and F refers to a non-linear function of ω r , r T , and the wind speed v w , where the detailed function F can be found in [41]. The stator currents i αs and i βs are calculated from (12), where the vector y can be expanded as the corresponding coefficient matrix C can be expressed as The continuous differential equations should be discretized before the numerical calculation. The corresponding time-discrete for (11) can be obtained after utilizing VOLUME 10, 2023 the trapezoidal rule: where t refers to the time-step of the EMT emulation utilized in the wind turbine, which is defined as 50 µs, and x(t + t) denotes the vector of state variables of next timestep.

C. MMC AVERAGE VALUE MODEL
The configuration of a MMC-based three-phase DC/AC converter integrated with the renewable energy is provided in Fig. 4 (a), where each phased contains 2N half-bridge submodules (HBSMs) and each arm has an arm inductor.
To avoid asynchronous data communication between BESS and the converter system, the EMT simulation is also applied in emulating the integrated DC/AC converter as well as the MMC stations in the microgrids with a time-step of 200µs. The hardware-based FTRT emulation prefers the models which require the least computational burden, and therefore, the average value model (AVM) for MMCs is utilized. The HBSMs can be simplified into the controlled voltages sources as given in Fig. 4 (b). When the upper switch K 1 is turned on, the submodule is inserted, while the submodule is bypassed when the K 2 is turned on. Assuming that the capacitor voltages are well balanced in the AVM, the average value of the capacitors and the equivalent voltage of an arbitrary submodule can be derived as.
where S i represents the switching function which yields 1 and 0 when the submodule capacitor is inserted and bypassed, respectively. As a result of the well-balanced condition in AVM, the circulating current can be neglected the AC side output voltage can be expressed as: where S ui and S li refer to the switching functions of upper and lower arms, respectively. The control strategy for the MMC model is similar to the inverter controller without the MPPT part. The MMC adopts a two-loop control scheme where the outer-loop controller is in charge of converter functions which regulate the active/reactive power and bus voltage, as given in Fig. 5 (a). The reference current i * d,q is calculated as: where the K p and K i are constants, V * and V refer to the control target reference and their feedback, respectively. The MMC current controller in d-q frame is provided in Fig. 5 (b), where the signals with the superscription * represent the reference values. The current controller amplifies the current error to obtain the voltage V d,q , which is converted into three-phase signals m abc that are sent to the MMC inner-loop controller employing phase shift strategy.

A. AC GRID MODELING 1) SYNCHRONOUS GENERATOR AND CONTROL SYSTEM
Transient stability simulation is basically solving a series of nonlinear differential algebraic equations (DAEs), which can represent the detailed dynamics of synchronous generators. Meanwhile, the network equations containing transmission lines and loads are given in (28), which calculate the non-generator bus voltages and generator output currents of the AC system.Ẋ where U refers to the vector of inputs such as field voltages (V fd ) and mechanical torque (T m ), X represents the vector of state variables of the synchronous machine. In this work, a detailed synchronous machine model is utilized, which contains two mechanical equations and four rotor electrical equations with 2 windings on the d-axis and 2 damping windings on the q-axis, and therefore, the vector X can be expressed as where δ refers to the rotor angle, and ω represents the derivative of angular velocity. Meanwhile, an excitation system with automatic voltage regulator (AVR) and power system stabilizer (PSS) is also included in the synchronous generator as given in Fig. 7, where v t refers to the terminal voltage of the synchronous machine, T R , K stab , T w , T 1 , and T 2 are constant parameters, which can be found in [43], and v 1 , v 2 , and v 3 are the intermediate variables in the excitation system, which are solved together with the mechanical and electrical equations. Therefore, the excitation system contributes 3 additional state variables in vector X and the 9 thorder differential equations are utilized for representing the synchronous generators of the proposed AC system, which are solved by the explicit 4 th -order Adams-Bashforth (AB4) given as: where h refers to the time-step of AC grid, which is defined as 1 ms.

2) AC NETWORK EQUATIONS
The AC network mainly contains the transmission lines, transformers, and loads. In order to reduce the computational burden during the hardware execution, the transmission lines and transformers are represented by the lumped π model. According to [43], only the fundamental frequency is considered, which means the other frequency ranges except for the system fundamental frequency between the microgrids and the AC network are ignored. The fixed loads VOLUME 10, 2023 and shunt compensators are integrated with the admittance matrix, given as where P Load and Q Load represent the active and reactive power of the load, respectively, and V pcc refers to the bus voltage at the point of common coupling. Following the introduction of the admittance matrix of the AC grid, the network equations in (28) can be expanded as where the subscript n refers to the generator nodes with current injection, and r represents the remaining nodes without synchronous generators. Due to the absence of current injection in the non-generator buses, the current vector of the remaining nodes I r = [0] and I n can be derived as Although the generator voltages V n are not directly known after solving the DAEs, the relationship between V n and I n can be expressed as where u 1−6 can be obtained following the acquirement of new state variables, which are given below.
u 2 = X ad sin 2 (δ) + X aq cos 2 (δ) + X l , Obviously, with the ascertained relationship between V n and I n , all the unknown vectors in (32) can be solved directly.

B. PROPOSED MICROGRID AND AC GRID INTERFACE
Since transient stability simulation is adopted for AC grid analysis, an interface is proposed so that it is compatible with the aforementioned EMT models. As given in Fig. 1  (d), the PV arrays are modeled as the current sources, while the wind turbine and BESS are modeled as voltage sources. As for the interface strategy, theoretically, there are three available strategies that can be applied, which are P-Q interface, voltage interface, and current interface. In the P-Q interface, the converter stations can be treated as time-varying loads in transient stability simulation of a traditional AC/DC grid, however, the admittance matrix needs to be updated every time-step, resulting in a heavy computational burden. The utilization of dynamic voltage/current injection interface avoids solving the admittance matrix in every time-step, which significantly increases the scalability of the proposed FTRT emulation. However, the current injection interface is not suitable for the proposed FTRT emulation. Since the synchronous generators are modeled as voltage sources and integrated with the network equations, the current injection interface will introduce additional calculations. Therefore, a voltage injection interface strategy is put forward in this work to maintain a constant admittance matrix and accelerate the hardware emulation.
Under this scheme, a converter is taken as a simplified synchronous generator with a varying voltage that does not have detailed electrical and excitation circuits. The reduced network equation (33) can be further expanded as where the matrix with subscription i refers to the generator node represented by the detailed Park's equations, m denotes the buses where the converter stations connected to, and i + m constitutes the n generator nodes in (32). Since different synchronous machines and converter stations are interconnected together in a transmission network, the variables should be expressed in a common reference frame, which is called a synchronously rotating D-Q frame. The above complex matrix equation (43) yields 4 real matrix equations in D-and Q-axis, given as where R and B represent the real part and the imaginary part of the corresponding Y matrix. The values of V Di and V Qi associated with the detailed synchronous machines are not directly known, but the voltages under the reference dq frame can be represented by the state variables following each step of integration, given as where R a , X l , X q1 , X q2 , X fd , X d1 , L ad , L fd , L d1 , L aq , L q1 , and L q2 refer to stator resistance, stator leakage reactance, reactance of damper winding q1, reactance of damper winding q2, field winding reactance, reactance of damper winding d1, daxis mutual inductance, field winding inductance, inductance of damper winding d1, q-axis mutual inductance, inductance of damper winding q1, inductance of damper winding q2, respectively, which are constant values of the synchronous 42 VOLUME 10, 2023 generator. Meanwhile, the variables with superscription represent the subtransient reactance and electromotive force. In order to correlate the components of the voltages and currents expressed in d-q frame to the rotating common reference D-Q of the network, the following reference frame transformation equations are used, where the variable δ in (52) and (53) represents the synchronous generator rotor angle given in (29). Therefore, by combining (48)-(53), the values of V Di and V Qi can be calculated by the new state variables. Following the solution of the microgrid functions, the components V Dm and V Qm are available for solving I Dm and I Qm in (46) and (47). Since distinct time-steps are adopted in TS and EMT emulation, the data synchronization of the proposed interface is as follows, the microgrid undergoing EMT emulation provides the voltages in D-Q frame to the AC grid in every 5 EMT time-steps, and the injected voltages are solved together with the network equations in the AC system. Then the calculated currents at PCC are in return sent to microgrid systems to continue the EMT emulation, as shown in Fig. 6.

IV. FTRT HARDWARE EMULATION OF GRID OF MICROGRIDS ON FPGAs
The full deployment of the AC system integrated with the microgrid cluster requires two Xilinx Virtex R UltraScale+ TM VCU118 boards containing XCVU9P FPGA which includes 6840 DSP slices, 1182240 look-up tables (LUTs), and 2364480 flip-flops (FFs). The host system as well as the microgrid-1 (MG-1) to MG-3 in Fig. 6 are deployed on the VCU118 Board-1, and the remaining four microgrids are implemented on the VCU118 Board-2. The sufficient hardware resources enable the MGs to be executed on the FPGA board in FTRT mode. The reconfigurability of FPGAs allows the hardware resources to be adjusted to accommodate and represent practical systems [44], and therefore it is suitable for the hardware-in-the-loop (HIL) emulation and dynamic security assessment in the energy control center. The hardware implementation can be conducted in two ways: utilizing hardware programming language, and block design. The former method models each circuit part in hardware language such as VHDL, which is complicated and requires hardware programming experience of the researcher. The Xilinx R high-level synthesis software Vivado HLS R is able to transform the C/C++ code into intellectual property (IP) which contains corresponding input/output ports in VHDL format. The circuit parts and subsystems which consist of the microgrid cluster are programmed in C/C++ code before the hardware design. After IP generation, each circuit component is converted to a hardware module and exported to Vivado R for block design,, which significantly increases the flexibility and reduces the complexity of hardware implementation. In a practical real-time application, data exchange is realized by connecting the input/output ports among the hardware modules. According to the correlation among the subfunctions, the hardware modules are designed to be calculated in parallel or series. The hardware block design and the data stream are provided in Fig. 8. After design synthesis and device mapping, the bitstreams were downloaded to the target FPGA boards via the Joint Test Action Group (JTAG) interface as given in Fig. 9. The Quad Small Form Pluggable (QSFP) interfaces are connected with cable and the build-in IP Aurora 66B/64B core is utilized to realize the data communication. Due to the little data required of the proposed AC/Microgrid interface, the PCC voltages and currents in D-Q frame are chosen as the communication data between the FPGA boards. Since the MG 1-3 are conducted in Board-1, the data exchange only happens inside Board-1. As for Board-2, the output voltages (V MG4−7,MMC,D , and V MG4−7,MMC,Q ) of MG 4-7 are delivered to the build-in IP Aurora Core through the AXI bus, then the voltage data is sent to Board-1 via the QSFP bidirectional cable. After solving the network equation in Board-1, the calculated currents (I MG4−7,MMC,D , and I MG4−7,MMC,Q ) are delivered to Board-2 through the QSFP cable and sent to the Aurora core. Finally, the current data is solved together with the VSC stations to keep the EMT simulation going on. Meanwhile, the output digital data is transferred to analog data via the digital-toanalogic converter (DAC) board, so that the waveforms can be displayed on the oscilloscope. Table 1 provides the hardware resource utilization and the latencies of the hardware modules in the FTRT emulation, where the Tclk in Table 1 refers to the unit of the latency. For example, the 39 Tclk means the latency of the PVmodel part is 39 clock cycles. As mentioned, the components in the microgrid including the PV array, wind turbine, and LVDC VOLUME 10, 2023    should wait for the wind turbine part to finish computation to keep data synchronization. Therefore, the total FTRT ratio of the proposed microgrid cluster is about 51.

V. FTRT EMULATION RESULTS AND VALIDATION
The hardware emulation of the AC grid integrated with microgrid cluster is conducted on the FPGA-based FTRT platform (Fig. 9), and the proposed FTRT emulation and the interface strategy are validated by comparing the results with those of the off-line simulation tool Matlab/Simulink R .

A. CASE 1: THREE-PHASE-TO-GROUND FAULT
At t = 5s, the three-phase-to-ground fault lasting 200 ms occurs at Bus 21 as given in Fig. 6. The rotor angles, output voltages, and frequencies of the synchronous generators start to oscillate immediately, as given in Fig. 10 (a)-(c), where the dashed lines refer to the results from Simulink R and the solid lines represent the FTRT emulation results. Fig. 10 (d) provides the output power of the VSCs in MG-1, which indicates that the waveforms from the FTRT emulation match with 44 VOLUME 10, 2023  those obtained from the off-line simulation. Since the output power of the PV-BESS systems and wind turbines maintain stability, there is no significant change of the active power injections at the PCC, as given in Fig. 10 (d).
To further demonstrate the accuracy of the proposed FTRT emulation, the electrical torques, generator output i d , and i q currents after the 200ms three-phase-to-ground fault are also provided in Fig. 11. As illustrated in the zoomed-in plots in Fig. 11. (c), the accuracy of the FTRT emulation can be guaranteed as the waveforms from the proposed emulation are matched with the results from the offline simulation tool.

B. CASE 2: MICROGRID INTERNAL POWER BALANCE
Since the output power of renewable energy is highly dependent on the environment, such as irradiations or wind speed, low power generation may occur and last for a long period in the microgrids. This case study focuses on the microgrid internal balance and the predictive control of the proposed FTRT emulation. At the time of 5s, the output power of each PV array in MG-1 decreases by 500 kW , which induces a  lack of power injection at Bus 39, as given in Fig. 12 (a). Although the rotor angles can restore to a new steady-state as given in Fig. 12 (b), the impacts of reduced generation are severe, including the significant drop of the generator voltages and the unrecoverable generator frequencies as shown in Fig. 12 (c) and (d).
With the 51 times faster than real-time execution, the FTRT emulation equipped in the energy control center comes up with an optimal power control strategy following the detection of the abnormal condition to mitigate the adverse impacts, as given in Fig. 13 (a). At t = 7.8 s, each BESS in MG-1 provides extra 1 MW active power to the AC grid in 1s and lasts 1.2s until t = 10.0 s, resulting in an extra active power injection of 2 MW from MG-1 to Bus 39. As a result, the frequencies start to recover as given in Fig. 13 (d). After 10s, MG-1 reduces the power injection from 4 MW to 3 MW , then the generator frequencies restore to 60 Hz, meanwhile, the rotor angles and generator voltages recover to the previous working conditions. The zoomed-in plots in Fig. 12-13 (d) indicate that the FTRT emulation results matched with the off-line simulation tool.

C. CASE 3: INTER-MG COORDINATION
In this case, assuming that the BESS in MG-1 is fully charged and cannot store the extra power generated from the PV array, so that the MG-1 injects 1 MW into the AC grid, as given in Fig. 14 (a). It brings significant impacts on the stability of the host grid, including the rotor angles, bus voltages, especially the increasing generator frequencies, as shown in Fig. 14 (b)-(d). After the detection of the abnormal frequency, the energy control center emulates several power control contingencies and provides an optimal solution to mitigate the increasing frequencies.
At t=8.0s, both MG-2 and MG-3 start to take action, and each microgrid absorbs 1 MW active power, As a result, the generator frequencies start to decrease as given in Fig. 15 (a) and (d). At the time of 15.2s, MG-2 and MG-3 reduce the absorption power to 0.5 MW , and the frequencies start to recover and return to 60 Hz. Meanwhile, the rotor angles and the bus voltages of the AC grids stabilize in a new steady-state as shown in Fig. 15 (b) and (c). Meanwhile, the zoomed-in plots are also provided in Fig. 14-15 (d), which thoroughly demonstrates the accuracy of the proposed method.

D. CASE 4: OPEN CIRCUIT FAULT IN LVDC SYSTEM
The open circuit fault occurring in the 5-T LVDC line in MG-1 is also emulated, where the fault location is shown in Fig. 6. The influence of the open circuit fault is similar to the low power generation fault due to the lack of power injection at Bus 39. The zoomed-in plots in Fig. 16 (b) demonstrate that the proposed FTRT emulation still has high accuracy regarding the fault happens at one of the LVDC lines.

VI. CONCLUSION
This paper proposed a comprehensive hardware-based fasterthan-real-time dynamic emulation of a grid of microgrids to study the impact of their integration on the host system. Since different emulation types and time-steps are adopted for the AC grid and microgrids, a dynamic voltage injection interface strategy with less hardware utilization and lower latency is therefore proposed to enable the compatibility of EMT and TS models. Taking its inherent advantages of reconfigurability and parallelism, the FPGA-based hardware platform allows emulating the integrated microgrid cluster with an execution speed over 51 times faster than realtime. Meanwhile, an active power dispatch is emulated to deal with the various working conditions of the microgrids. The time-domain results indicate that the proposed FTRT emulation is numerically stable and accurate compared with the off-line simulation. Furthermore, the dynamic security assessment can be carried out on the proposed FPGA-based FTRT platform with significant acceleration. The detailed comprehensive FTRT emulation is also suitable for analyzing other severe disturbances, which is meaningful for modern power systems with high penetration of renewable energy.