Evaluation of Interoperable Distributed Energy Resources to IEEE 1547.1 Using SunSpec Modbus, IEEE 1815, and IEEE 2030.5

The American distributed energy resource (DER) interconnection standard, IEEE Std. 1547, was updated in 2018 to include standardized interoperability functionality. As state regulators begin ratifying these requirements, all DER—such as photovoltaic (PV) inverters, energy storage systems (ESSs), and synchronous generators—in those jurisdictions must include a standardized SunSpec Modbus, IEEE 2030.5, or IEEE 1815 (DNP3) communication interface. Utilities and authorized third parties will interact with these DER interfaces to read nameplate information, power measurements, and alarms as well as configure the DER settings and grid-support functionality. In 2020, the certification standard IEEE 1547.1 was revised with test procedures for evaluating the IEEE 1547-2018 interoperability requirements. In this work, we present an open-source framework to evaluate DER interoperability. To demonstrate this capability, we used four test devices: a SunSpec DER Simulator with a SunSpec Modbus interface, an EPRI-developed DER simulator with an IEEE 1815 interface, a Kitu Systems DER simulator with an IEEE 2030.5 interface, and an EPRI IEEE 2030.5-to-Modbus converter. By making this test platform openly available, DER vendors can validate their implementations, utilities can spot check communications to DER equipment, certification laboratories can conduct type testing, and research institutions can more easily research DER interoperability and cybersecurity. We indicate several limitations and ambiguities in the communication protocols, information models, and the IEEE 1547.1-2020 test protocol which were exposed in these evaluations in anticipation that the standards-development organizations will address these issues in the future.


I. INTRODUCTION
Around the world, Distributed Energy Resource (DER) grid codes and interconnection standards have been evolving to include new grid-support functions. These standards often include functionality to provide voltage regulation (e.g., fixed power factor, voltage-reactive power, voltage-active power, and active power-reactive power functions), frequency response (frequency-active power or frequency-droop), The associate editor coordinating the review of this manuscript and approving it for publication was Yuh-Shyan Hwang . and voltage and frequency ride-through capabilities [1]- [3]. Historically, these capabilities are enabled during the manufacturing or commissioning process and left to operate autonomously for the life of the equipment. While some standards have included ranges of adjustability for the functionssuch as in California Rule 21 [4] and the Australian/New Zealand interconnection standard AS/NZS 4777 [5], [6]-the US interconnection standard, IEEE 1547-2018 [7], is the first to require DER equipment include ranges of adjustability and mandate DER devices include a standardized communication interface. This interface is designed to enable grid operators VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ to read DER measurements and adjust settings and operating modes in near real-time.
Communications-enabled DER provide a wide range of benefits to grid operators. Transmission and distribution system operators have better visibility into distribution networks and can optimize DER settings according to schedules or real-time needs. There have been many studies that show the advantages of using DER Management Systems (DERMS) to configure the operations of fleets of DER devices. For example, carefully tuned voltage-reactive power (volt-var) curves or power factor set points can increase feeder hosting capacity [8] and support voltage on unbalanced feeders [9]. Communication-enabled DER also allow for new voltage regulation optimization techniques-e.g., Particle Swarm Optimization and Extremum Seeking Control-to be developed to set DER reactive power set points to minimize voltage deviations and circuit losses [10]. At the bulk system level, it is also possible to select frequency-watt parameters to provide fast frequency reserves [11], [12] and wide-area damping [13].
DER communications will also open up markets to provide third-party virtual power plants (VPPs) or aggregation services. In the United States, Federal Energy Regulatory Commission (FERC) Order 2222 [14] was passed in September 2020 which allows DER aggregations to participate in regional wholesale markets. In combination with DER interoperability functions, the FERC order will pave the way for VPP players in Regional Transmission Organizations (RTOs) and Independent System Operators (ISOs) wholesale markets in the U.S. There has been substantial research in different VPP approaches to use the DER active power and curtailment set points to provide transmission services. As an example, Castillo et al. designed a stochastic risk aversion-based rolling horizon optimization framework to bid into day-ahead scheduling and real-time dispatch markets [15]. Others investigated VPP optimization using chance-constrained methodologies [16], robust optimization [17], and stochastic programming [18] to meet load or generation needs.
In order to provide the aforementioned grid services, DER equipment must operate and communicate as expected. Unfortunately, it is challenging to verify DER equipment is functioning in accordance with grid codes and standards [19]-taking weeks if not months to perform all the experiments. In the last decade, evaluating DER equipment has been the source of significant academic interest and corporate investment. In the U.S. and Canada, DER equipment must be listed in accordance with regional requirements where it is installed. In most states, the requirements are harmonized to IEEE 1547-2003. The step-by-step test procedure for evaluating DER equipment to IEEE 1547 requirements is documented in IEEE 1547.1, but Nationally Recognized Testing Laboratories (NRTLs) certify DER compliance using Underwriters Laboratories (UL) 1741 [20]-which heavily references IEEE 1547.1 but provides greater testing detail in some key areas. UL 1741 was used to certify DER  While some DER devices may include a DNP3 or IEEE 2030.5 interface natively, Modbus is currently the prevailing technology on the market. However, Modbus does not include any encryption or authentication, 1 so it is not intended to leave DER facility premises and will likely only communicate a very short distance to a IEEE 2030.5 or DNP3 gateway or converter-that may be a DER bolt-on module like that demonstrated in a California Solar Initiative project led by EPRI [35]. Using a converter, the DER will be able to connect to utility or aggregator systems through the public internet with standardized cybersecurity protections.
In this work, we validate the interoperability of multiple DER simulators using the information models in Table 1 in accordance with the IEEE 1547.1 compliance test protocol. This process exposed a number of errors, ambiguities, and issues with the test protocol, information models, and DER simulator implementations. These findings have been relayed back to the appropriate standards-making bodies and stakeholders. This paper represents the first detailed investigation of these information models using the DER interoperability certification procedure and is the first to demonstrate the IEEE 1547 communication protocols. The remainder of the paper is structured as follows: Section 2 describes the methodology used to conduct the experiments; Section 3 provides detailed results of the experiments; Section 4 summarizes the standards development recommendations; and Section 5 concludes the paper with the major findings from this work.

II. EXPERIMENTAL APPROACH
The SVP was designed as a highly versatile platform for automating certification experiments. Now, more than eight years in development, it includes a wide range of abstraction layers and equipment drivers. The abstraction layers are used to expose different classes of drivers in the GUI so the same testing logic (scripts) can be used to run experiments at different laboratories by only changing the test parameters in the script [22], [39]. Current abstraction layers included in the SVP Energy Lab repository [40] are AC/grid simulators, DC/PV simulators, DER, battery simulators, data acquisition systems, and gensets, switches, and loads. Using this software, Sandia created an IEEE 1547.1 interoperability certification script [41] that automates the evaluation of DER communications for nameplate, configuration, and setting information. This script also provided a means to spotcheck the communications required to run all the electrical tests that can be configured via the communication port.
Executing a full IEEE 1574.1 interoperability compliance test for a given communication protocol is challenging because it requires verification of the entire information model. The test procedure is shown in Table 2. To certify a device to any of the protocols, in addition to verifying the measurement points by adjusting the grid simulator settings or DER operating modes, the DER must be evaluated for the management functions (volt-reactive power, freq-droop, etc.) using the specified communication protocol. The pass/fail criteria for those tests is the same as the management function tests which evaluate the electrical characteristics of the DER. In order to accelerate and automate IEEE 1547.1 testing, the SVP was updated to include drivers for SunSpec Modbus via pySunSpec2 [42], IEEE 1815, and IEEE 2030.5. To conduct the nameplate, configuration, and monitoring information tests, a IOP.py SVP script was created. Additional IEEE 1547.1 SVP scripts were also created to evaluate the interoperability and electrical functionality of the management functions, as shown in the final column of Table 3.
Sandia, SunSpec, EPRI, Kitu Systems, and SIRFN labs collaborated to create the testing scripts and DER simulators required to evaluate the conformance tests and information models. The SVP test environment used for these evaluations is shown in Figure 1. The SVP was connected to four DER end-point simulators which each used an IEEE 1547mandated protocol: • SunSpec DER: A SunSpec-constructed DER emulator that ingests a JSON file to create a SunSpec-compliant device with all of the 700-Series Models. This device did not include a power system or power electronics simulation capability-only a state-based representation of a SunSpec-compliant DER device. The SVP interacted with this DER simulator using the pySunSpec2 python package.
• IEEE 1815 DER: The EPRI DER Simulator version 1.0.6 with a DNP3 interface included a partial implementation of the DNP3 Application Note (App Note) AN2018-001. This simulator is a windows executable with the ability to run the power electronics simulation with irradiance, grid voltage, and grid frequency temporal profiles loaded from CSV files [43].   with an IEEE 2030.5 CSIP information model with an optional DER simulator. The private keys for the Kitu Client were used to communicate to the device using an IEEE 2030.5 server developed by SunSpec. This server configured default settings (as opposed to scheduled behaviors) and was a standalone application from which the Kitu client retrieved information at fixed poll rates.
• IEEE 2030.5 DER #2: A IEEE 2030.5-to-Modbus converter developed by EPRI that interfaced with the EPRI DER Simulator. This converter ran as an executable on Ubuntu 20.04.2 LTS with a configurable poll rates. The converter was also configured to connect to the SunSpec IEEE 2030.5 server as shown in Figure 1. In the case of the IEEE 2030.5 experiments, the SunSpec server was used because it allowed the greatest flexibility and visibility in debugging and viewing client-server errors and messages. A range of other client-server topologies could have been investigated-and should be in the future to ensure true interoperability. For instance, the Kitu Client and EPRI converter could also connect to a Kitu IEEE 2030.5 Server, the IEEE 2030.5 Test Tools from QualityLogic, or other commercial IEEE 2030.5 servers. To perform the interoperability experiments with that equipment, the SVP would need the ability to update server configurations and retrieve client data from the Kitu Systems NorthGate Server API or QualityLogic test environment.
An added complication with IEEE 2030.5 testing was provisioning the SVP with the client certificate and private key to establish the Transport Layer Security Version 1.2 (TLS 1.2) session with the server. The Common Smart Inverter Profile (CSIP) defines a specific cipher suite (TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8) to perform asymmetric encryption of IEEE 2030.5 exchanges [44]. In order to establish the connection and interact within the Public Key Infrastructure (PKI) ecosystem, 2 the IEEE 2030.5 server was configured with the server certificate, server private key, and root certificate in Privacy Enhanced Mail (PEM) files. The server made the TLS connection with OpenSSL to the Kitu and EPRI clients using those PEM files, the certificate and private key of the client, and the cipher specifications.
It should also be noted that the IEEE 1547.1 testing is not a comprehensive interoperability test sequence. It is designed to verify a basic level of functionality to demonstrate the DER communication interface is connected appropriately to the electrical control and measurement capabilities of the DER. In order to fully validate the communication capabilities of DER, a separate certification program has been established by the SunSpec Alliance for IEEE 2030.5 clients and servers, 2 The SunSpec Test PKI certificates were used for all the tests. Details on the certificates are provided in [45]. SunSpec Modbus devices, and IEEE 1815 masters and outstations [46]. In the program, vendors submit their products to one of the SunSpec Authorized Test Labs (ATLs) for compliance certification. Those labs verify the full communication functionality of the devices and send the trace files to SunSpec for final certification validation. For instance, in the case of the Modbus devices, this process verifies that single registers can be read/written, multiple adjacent holding registers can be read/written using a single request, and multiple non-adjacent holding registers can be read/written in multiple requests [47]; and in the case of IEEE 2030.5, the program verifies the security features (e.g., appropriate cipher suites, certificate encoding, establishing valid TLS 1.2 connections, etc.), uses the HTTP protocol, checks the function sets (time, event handling, event priorities, etc.), and verifies that all the IEEE 1547-mandated functions can be acquired from the server and are implemented in the client [48]. These experiments are not included in the IEEE 1547.1 test procedure.

III. RESULTS
Working with these prototype DER devices, the first handson assessment of the IEEE 1547.1 interoperability tests were conducted. Unfortunately, not all the functionality was included in these devices (e.g., statuses, alarms, operational states, etc. we're not present in the EPRI simulator), so not all the functionality was evaluated for each of the protocols/DER simulators. To execute the experiments, a DER 1547 abstraction layer was created and read and write methods were built for the drivers for IEEE 2030.5 (der1547_ 20305.py), IEEE 1815 (der1547_dnp3.py), and SunSpec Modbus (der1547_sunspec.py). The following section presents the results from the experiments. Errors and ambiguities in the test and communication protocols were identified for feedback to the standards working groups.
Each of the DER simulators and associated communication tools were instantiated at the beginning of the IEEE 1547 interoperability tests. In the case of the SunSpec Modbus, a JSON mapping of a SunSpec Modbus device was imported by the pySunSpec2 Python package and this Sun-Spec object was used for all the interactions. This object did not include a power electronics model so the measurements were static values included in the JSON data file.
For the IEEE 1815 experiments, the SVP used an operating system bash shell command to start the DNP3 Agent with variables to indicate the Agent IP address and Outstation IP address (both use the 127.0.0.1 loopback address since the SVP, DNP3 Agent, and EPRI DER Simulator are all on the same Windows 10 machine), the Agent and Outstation IP port, Outstation local address, Master local address, and request ID. The EPRI Simulator was also started with a system command from Python and pywinauto was used to interface with the GUI. The DER nameplate capacity information was preloaded as a CSV in the Windows GUI. At start up, the SVP enabled the DERMS mode in the EPRI Simulator and the DNP3 Agent connected to the outstation and read the AI and BI points in the device. Then, the SVP commenced the IEEE 1547.1 test sequence with the EPRI Simulator outstation using the DNP3 Agent API to issue read/write commands to the device. VOLUME 9, 2021 For the IEEE 2030.5 testing, the IEEE 2030.5 Server was started on port 9443 for the Kitu experiments and port 8443 for the EPRI converter experiments. The clients were then initialized and communicated to the server by getting /sep2/dcap, creating a resource tree with dcap, tm, der, edev, fsa, derp, dc, derc, dderc, rsp, upt, and mup paths for the given Short Form Device Identifier (SFDI). The SVP communicated to the SunSpec server API via an HTTP API on the loopback IP address. Initially, there were challenges communicating with the EPRI converter because the server was closing the HTTP connection between messages and the client expected the connection to remain open for the duration of the experiments. Since a TCP connection can be closed for many reasons outside of the control of the endpoints, it is important to ensure communication failures are recoverable. This highlights some of the interoperability challenges that exist when testing with a single client or server-there will be different use cases that are not represented by the IEEE 1547.1 requirements.

A. NAMEPLATE DATA TEST
The nameplate data provides information about the DER device. The IEEE 1547.1 test procedure requires the tester to read the values for each of the nameplate data points and compare them against the manufacturer-provided values.

1) SunSpec MODBUS
The nameplate data is included in SunSpec Common Model 1 and DERCapacity Model 702, as shown in Table 4. In the case of the simulated SunSpec-compliant DER, the IEEE 1547-mandated points were read as anticipated. Notably, there are additional nameplate data points included in Sun-Spec Model 702, added based on the needs of the stakeholders in the SunSpec Alliance Modbus Working Group, but these were not required for IEEE 1547 certification.

2) IEEE 1815
With the exception of Manufacturer, Model, Serial Number, and Version Data, the DNP3 App Note includes AI and BI points for all the Nameplate data points as shown in Table 4 and aligned with IEEE 1547.1 and App Note Table 63. Manufacturer, Model, Serial Number, and Version Data are optional Device Attribute Objects in point number 0 of object group 0. So in practice, the master can read Device Attributes -Device serial number at Group 0 Variation 248, Device Attributes -Device manufacturer's product name and model at Group 0 Variation 250, and Device Attributes -Device manufacturer's name at Group 0 Variation 248 with point index 0, though the EPRI Simulator did not include this functionality. In fact, the DNP3 App Note requires only a DNP3 Level 2 (DNP3-L2) device so some DERMS masters many not have the ability to read this class object data. Therefore, it is recommended to move these values to AI points in the DNP3 Application Note.
Not all the other data points were included in the EPRI DER Simulator either, e.g., Active power rating at specified over-excited power factor, Abnormal operating performance category, and some of the Supported control mode functions returned null, but the DNP3 driver and DNP3 Agent successfully returned numeral and string data and reporting null data points to the user, demonstrating the ability of the SVP and DNP3 Agent to complete the IEEE 1547.1 Nameplate Data Test.
3) IEEE 2030.5 The Kitu Client was configured with a 600-second poll rate and the EPRI converter with a 60-second poll rate, after which they retrieved the control resources associated with the client from the server. The SVP configured the topological and DER resources in the server using the SunSpec backend API in order to establish the client-server connection and program IEEE 2030.5 settings. The information pushed to the server was nameplate information, monitoring data, and status/alarm information. This information was stored in the server and pulled into the SVP via the API. Each class of resources includes a poll rate, so the monitoring data could be updated at a quicker rate than status information, for example. The DER nameplate information is POSTed to the server a single time after GETting down the end device resource locations from the server. The SVP gets the DERList link based on the SFDI and Long Form Device Identifier (LFDI), finds the DeviceInformation and DERCapability links, and then reads the nameplate points. The DERCapability Link and Resource are shown in Fig. 7 with the DERSettings and DERStatus points.
The paradigm for testing a IEEE 2030.5 client is much different than Modbus and DNP3 because the SVP is not directly reading or writing data on the DER. The SVP is configuring the server and then waiting for the client to interact with it. As a result, there are much longer test times because the SVP must wait (up to the resource poll rate) for the client to exchange data with the server.

B. CONFIGURATION DATA TEST
For the configuration information tests in IEEE 1547.1 the following parameters are read and the DER behavior is measured through a data acquisition system: • Active power rating at unity power factor (nameplate active power rating) • Apparent power maximum rating • Reactive power injected maximum rating • Reactive power absorbed maximum rating • For ESS, active power charge maximum rating • For ESS, apparent power charge maximum rating These parameters are set to 80% of the initial value, re-verified with the data acquisition system, and returned to 100% of the initial value to be verified a final time. The pass/fail indicates that the values should match the configuration data. Adding an allowable tolerance on the accuracy of these points would be useful in the future. The Supported control mode functions are also verified to operate as expected when enabling and disabling each of the management (grid-support) functions; although no specifics are provided on how to do this, leaving it to the interpretation of the test engineer on how to perform the experiment. Adding more detail in IEEE 1547.1 to that test case would be helpful to ensure all the experiments are conducted in the same manner.
In IEEE 1547.1, the SunSpec, DNP3, and IEEE 2030.5-specific tests state that the nameplate points are to be overwritten for the configuration information data tests. This is a poor practice, and SunSpec Modbus and IEEE 2030.5 have created settings points so that the nameplate data is never overwritten. There are also typos in IEEE 1547.1 mislabeling IEEE 2030.5 nameplate and configuration information test section titles (Sections 6.8.2.1 and 6.8.2.2), which should be cleaned up in the next revision.

1) SunSpec MODBUS
Using pySunSpec2 to interact with the setting points in the DERCapacity Model produced the intended changes in the SunSpec Models. Since this DER simulator did not include a power simulation, there was no way to independently verify the DER operations were changed with these updates. In the future, this will need to be verified for DER devices when undergoing IEEE 1547.1 tests.

2) IEEE 1815
As stated above, the IEEE 1547.1 standard states the tests should use the same data points for configuration information as the nameplate data points. In the case of DNP3, this is not possible because nameplate data are AI and BI points, as indicated in Table 63 of the DNP3 Application Note. As a result, there are no DNP3 outputs (DER inputs) that permit changing the nameplate information. The DNP3 App Note and IEEE 1547.1 should be updated with AO and BO points to adjust the nameplate values. Or, preferably, there should be a completely new set of AO/BO and AI/BI points that represent the device settings-similar to the approach used in SunSpec Modbus and IEEE 2030.5. This way the nameplate DNP3 information can never be changed. This is a significant gap in the DNP3 information model that will need to be updated to allow DER devices to be tested to the IEEE 1547.1 requirements.

3) IEEE 2030.5
While IEEE 1547.1 states that the DERCapability Resource should be used to make configuration data changes on the client-side and IEEE 2030.5 Appendix A states that HTTP GET/HEAD and PUT are mandatory for DERCapability, it is preferred to use DERSettings to update the configuration parameters in the client. As shown in Figure 7, DERSettings includes setMaxW, setMaxVA, setMaxVar, setMaxVarNeg, setMinPFOverExcited, setMinPFUnderExcited, setMaxChargeRateW, setMaxChargeRateVA, and mod-esEnabled which allows the SVP to communicate the configuration changes to the client/DER via the server. An example XML exchange to make this update from the server is shown at the bottom of Fig. 2.
Generally, the client will PUT their DERCapability and DERSettings to the server to inform the server of the DER nameplate ratings and settings. DERCapability and VOLUME 9, 2021 FIGURE 2. The client-server interaction to gather nameplate information and to update the client settings.
DERSettings are not meant for the server to make changes to these values on the client. Instead, in the case of the Kitu system, there was a subset of client ''settings'' that a server can change on the client. These changeable settings are mapped to a DefaultDERControl (e.g. setGradW, setSoftGradW, and Enter Service settings). If the server sets any of these Default-DERControls, the client will change its DERSettings and put the new values to the server. During the tests, the server DER-Settings configuration was updated with the new DERSettings data and read back using the API; however, it was much more difficult to evaluate if the client picked up these settings or taken affect. The IEEE 1547.1 test requires that the test lab verify ''the value reported matches the behavior of the DER measured through independent test equipment separate from the DER interface''. To do this, the SVP is designed to adjust the settings, wait for the poll rate, read back the setting in the server and then independently check the power measurement using a data acquisition system. It is not clear how to verify supported modes, other than to run electrical tests which would indicate their operation when enabled and disabled-a time intensive and difficult task that is not defined in IEEE 1547.1.

C. MONITORING INFORMATION TEST
The IEEE 1547.1 monitoring information tests are designed to verify the DER can measure and report grid conditions, internal states, and alarm statuses. The tests set the DER to two operating points shown in Table 6, measured the value after at least 30 seconds, and verify the reported values match the operation conditions. The pass/fail criteria in 6 states the values should be within the allowable accuracies for each of the measurements in IEEE 1547 Table 3. The test procedure does not indicate how to produce the operating points, so it may be helpful for the IEEE committee to clarify this in the future. For instance, if testing a PV inverter, it would be possible to produce the active power setpoints be either changing the DC input to the EUT or commanding the DER to an active power curtailment mode. While it may be beneficial to use external sources to produce these operating points as to not rely on DER controls to validate the monitoring information, to produce the reactive power setpoints, a reactive power mode will need to be used. Oddly, in the general IEEE 1547.1 interoperability testing section for monitoring (Section 6.6.2) does not include the Operational State of Charge (SoC) that is in the protocol-specific requirements in Sections 6.8.1.3, 6.8.2.3, and the DNP3 App Note Table 63, shown in the monitoring data points in Table 5.
There is poor alignment between the information models in terms of data points or naming conventions for DER states and alarms. The reported data required for the Operational State test is an on/off indication; the connection status data must report a connected/disconnected state, and only a single alarm is required for the Alarm Status. There is no direct mapping between the information models shown in the comparison of SunSpec Modbus 701.Alrm, the DNP3 App Note Alarms in BI0-BI9, and the IEEE 2030.5 CSIP alarms from the bit-mapped resource, DERStatus:alarmStatus, as shown in Table 7. As a result, it is very difficult to create a communication protocol agnostic test for the alarms. Currently, the grid voltage in the IOP test is set to 1.25 * V nom which will cause a high voltage alarm for SunSpec and IEEE 2030.5 devices, but it is not clear what, if any, alarms will be raised on a DNP3 device. Poor harmonization between the information models also makes creating an EUT abstraction layer difficult because all alarm names have to be supported. For testing purposes, these status/state options were converted to a binary value to autonomously issue pass or fail results for the monitoring tests.

1) SunSpec MODBUS
The SunSpec Modbus simulator did not include a power electronics simulation, inter-register logic/state machine, or have the ability to measure a power system, but these experiments were conducted to demonstrate the test sequence. To test active power measurements, the Limit Active Power (LAP) function was used to create the two test conditions. Constant Reactive Power (CRP) was used to test the two reactive power monitoring points-although the SVP test script does offer the option to test with Constant Power Factor (CPF). In creating the test script, it was discovered the Injected/Absorbed indications in Reactive Power (Injected) and Reactive Power (Absorbed) operating points were ambiguous and could indicate the active power direction or the excitation of the reactive power. It was assumed to be the latter. The voltage, frequency, operation state, connection status, and alarm status were measured and manually verified to reflect the state of the DER simulator.

2) IEEE 1815
The DNP3 interface on the EPRI DER Simulator demonstrated the ability to monitor active power, reactive power, voltage, frequency, and connection status. The active power was verified using LAP; reactive power was inspected using CPF; and the connection status checked with a connect/disconnect command written to BO5. The DNP3 voltage and frequency monitoring points were confirmed by manually adjusting the voltage and frequency sliders on the DER Simulator GUI. The operational state and alarms were not implemented in the DER Simulator, so those DNP3 points and associated call in the DNP3 driver could not be validated.

3) IEEE 2030.5
Using the MirrorMeterReading XML schema in Fig. 3, the IEEE 1547.1 measurements points were transferred to the IEEE 2030.5 server. This action was completed at the meter post rate defined in the client. This occurred at 300 second update rates for the Kitu client and 60 seconds for the EPRI client. The SVP queried the server every second until the server was updated to a monitoring information value within the permitted range. This process took much longer than the DNP3 and SunSpec Modbus tests because of the added time waiting for the client to send a measurement update.

D. MANAGEMENT INFORMATION TESTS
The management information tests cover the management functions included in Table 3: constant power factor, voltagereactive power, active power-reactive power, constant reactive power, voltage-active power, voltage trip, frequency trip, frequency droop, enter service and cease to energize and trip, limit maximum active power. These are the functions that include adjustable settings in IEEE 1547-2018. To test them, the test sequences from the electrical type tests were repeated using the standardized interface to communicate the settings to the EUT. Originally, there was no mechanism to enable or disable the anti-islanding functionality of the DER in the information models, which is required for the IEEE 1547.1 Unintentional Islanding type tests. In the model review process, SunSpec adopted the Anti-Islanding Enable (AntiIslEna) point in the DER AC Controls Model. It is recommended this point also be added to IEEE 1815 and IEEE 2030.5 because it will provide a standardized means to conduct Unintentional Islanding experiments.
The SIRFN community is working to construct the IEEE 1547.1 tests as described in the Introduction and listed in Table 3, but it was desired to spot-check these functions in the interoperability SVP script in order to assess if there were any issues with the information models or DER simulators. The following experiments were conducted: Note: the test criteria is for Enter Service electrical type testing, which does not include any trip testing, so this should be relabeled as to not indicate that it is a trip test. • Limit Maximum Active Power (LAP): SVP enabled the function and set the limit to 25%, 59%, 87%, and 45% of nameplate %WMax capacity, checked the active power monitoring point for changes, and disabled the function. Results and discussion are provided below for each of the protocols.

1) SunSpec MODBUS
While completing these experiments, the SunSpec Modbus 700-Series Models were in TEST status and actively being updated by the committee so it was relatively straightforward to report suggested changes and have those modification made quickly. In this process, a range of issues at the SunSpec model definition, pySunSpec2, SVP Dashboard, and SVP-levels were resolved. Some Modbus points were added and removed. There were missing labels for some points. Scaling/rounding issues were fixed. Point names, data types, units, and enumerations were changed. After these modifications, the SunSpec models and DER simulator functioned as expected during the management function spot checks. Each of the settings and management function parameters were able to be written and read back effectively. To help see the Modbus parameter and interact with DER equipment, the SunSpec Alliance has created a windows program called the SVP Dashboard that allows a user to quickly read a SunSpec Modbus map from a device and make changes to the parameters using a web browser. A screen shot of this tool with the voltage-reactive power model displayed is shown in Fig. 4. Since the SVP Dashboard was constructed using the pySunSpec2 Python library like the SVP driver, the python object interactions were the same with the simulated DER. Both of these tools will need to be tested against physical DER devices that have Modbus TCP and Modbus RTU implementations to confirm the full functionality in the future. Using the SVP to automate the IOP experiments, the nameplate, configuration, monitoring, and management experiments were completed in 3 minutes 24 seconds, with a 1 second delay between write and read test steps.

2) IEEE 1815
As described before, not all the IEEE 1547-2018 functionality was included in the EPRI DER Simulator, but it did include many of the grid-support functions. The spot check on CPF confirmed that in DER generating mode overexcited and underexcited PF values could be configured and read back from the DER using the DNP3 interface. To change DNP3 curve-based functions, first the Curve Edit Selector (AI328/AO244) was used to select the curve, set the Curve Mode Type, e.g., VV, VW, etc. (AI329/AO245), number of points (AI330/AO246), the X and Y Units (AI331/AO247, AI332/AO248), and then read/write up to 100 curve point values. The Volt-Var curve index was written to AO217 to indicate which curve should be used when the function was enabled with BO29. This process was performed to store the curve. While it may be necessary for production devices to write specific curves for each of the functions, for the IEEE 1547.1 testing, all the functions used curve index 1, so that implementation errors could be quickly identified. Although, testing the storing and recalling capabilities of the DER would be necessary to certify the device for DNP3 App Note compliance.
VV, WV, CRP, VW, FW, and LAP tests were successful. During the LAP experiments, a packet capture was performed on the loopback interface using Npcap and Wireshark, as shown in Figure 5. The communication between the SVP and the DNP3 Agent were filtered out of the figure to just show the DNP3 traffic between the DNP3 Agent on port 11949 and the DNP3 outstation on port 20000. Packet 395 sets the active power level of the DER to 25% by writing AO88. There is a scaling factor in the DER that converted the 250 value to 25. Packet 435 was the direct operate command which writes a 1 to BO17 to enable the LAP function. The measured active power (AI537) and the Limit Active Power Mode setting (BI69) were requested from the outstation in packet 509, and returned to the master in packet 511. The AO writes set the active power level to 59%, 87% and 45% in packets 629, 880, and 951. The active power measurements from AI537 occur after each change. After issuing each of the active power curtailment commands, the EPRI DER simulator reduces the power output, which is reflected in the GUI and in the measured power point. A screenshot of the DER EPRI Simulator GUI interface after setting the 45% LAP command is shown in Figure 6. The vertical yellow line labeled Wmax in the Active-Reactive Power (P-Q) plane represents the LAP reduction in power. This simulated device was configured to represent a 250 kW PV inverter, so the 45% curtailment changed the power output to 112.5 kW, even though there was 100% DC power input available to it.

3) IEEE 2030.5
In the case of the IEEE 2030.5 spot checks, the SunSpec Server API interface was used to issue a number of changes to the Kitu and EPRI clients. For these experiments the default settings in DefaultDERControl (dderc) were changed for each of the management information points (e.g., gridsupport function parameters) in the SunSpec Server. The settings were then read back from the server. This is not a sufficient experiment to confirm the client or DER has updated the settings-but there is no way to do this with the current IEEE 2030.5 protocol. Fortunately the IEEE 1547.1 test procedure requires the electrical tests for each of these modes, to show that the client effectively updates operations. But grid operators will have to assume that the client has updated its control settings in the field. Additionally, there is no clear way to ''disable'' an operating mode in IEEE 2030.5. The only option is to set the control points to None, but it is ambiguous whether the client will disable that control mode or leave the default settings running in those cases.

4) GRID OPERATIONS
In additional to testing challenges, there are also a number of operational concerns that were identified through the Management Information Tests. These issues are the result of permitting multiple valid IEEE 1547-2018 interoperability implementations. From the prospective of the grid operator, they would like to send a single command to all devices and have a deterministic response. However, the optionality of some of the Management Information parameters currently makes this impossible. Two examples of this are: • Do all DER devices need to support six points to create the P/Q and P /Q WV curves? In the case of a mix of four-and two-quadrant DER devices, the grid operator would want to send a single command that would include the P /Q points, but this command may be rejected by DER without storage. Therefore, it is VOLUME 9, 2021 FIGURE 5. A packet capture of the DNP3 LAP experiment. AO88 is the active power curtailment value, BO17 is the limit active power enable/disable point, and AI537 is the active power measurement point.
recommended that all DER equipment support those points and, for those that do not include storage, P /Q points are ignored.
• DER devices historically do not support all the reactive power units (e.g., %WMax, %VarMax, %VarAval, or %VAMax) and will generate an exception if they are commanded into those modes. • IEEE 1547-2018 -The standard should be updated to clarify what curves DER devices support in order to promote broad interoperability. This includes mandating DER equipment support six points that represent the P/Q and P /Q IEEE 1547 WV curves. -IEEE 1547 should be updated to require DER include a preferred, standardized unit for reactive power, preferably %VAMax, in order to reduce the complexity of grid operator systems and minimize the chances of misoperation.
• General Information Model Recommendations -There is poor alignment between the information models for DER states and alarms, e.g., SunSpec Modbus 701.Alrm, the DNP3 App Note Alarms in BI0-BI9, and the IEEE 2030.5 CSIP alarms from the bit-mapped resource, DERStatus:alarmStatus. The standards development organization should work to harmonize the DER states and alarms.
• SunSpec Modbus -Through the course of this research, a number of issues were identified in the SunSpec VOLUME 9, 2021 Modbus 700-series models and pySunSpec2. This work was performed while the SunSpec models were in TEST status, so those issues could be logged as GitHub issues, and brought to the committee for revision. These included missing points, naming conventions, selection of data types, problems with scale factors, etc. Each were addressed by the SunSpec Models Working Group before going to APPROVED status in April 20, 2021. . This test script was executed against DER simulators running each of the protocols. In this process, the team unearthed multiple issues with the IEEE 1547.1 test procedure, the information models, pySunSpec2, and the DER simulators running each of the protocols. These issues have been raised with the appropriate companies and committees to address these concerns and streamline the roll-out of advanced inverters. The SVP can also be used by DER vendors and nationally recognized testing laboratories to efficiently complete these experiments, thereby reducing the costs for developing these new products and listing them to the revised IEEE 1547.1 standard.