Cybersecurity Resilience Demonstration for Wind Energy Sites in Co-Simulation Environment

Sandia National Laboratories and Idaho National Laboratory deployed state-of-the-art cybersecurity technologies within a virtualized, cyber-physical wind energy site to demonstrate their impact on security and resilience. This work was designed to better quantify cost-benefit tradeoffs and risk reductions when layering different security technologies on wind energy operational technology networks. Standardized step-by-step attack scenarios were drafted for adversaries with remote and local access to the wind network. Then, the team investigated the impact of encryption, access control, intrusion detection, security information and event management, and security, orchestration, automation, and response (SOAR) tools on multiple metrics, including physical impacts to the power system and termination of the adversary kill chain. We found, once programmed, the intrusion detection systems could detect attacks and the SOAR system was able to effectively and autonomously quarantine the adversary, prior to power system impacts. Cyber and physical metrics indicated network and endpoint visibility were essential to provide human defenders situational awareness to maintain system resilience. Certain hardening technologies, like encryption, reduced adversary access, but recognition and response were also critical to maintain wind site operations. Lastly, a cost-benefit analysis was performed to estimate payback periods for deploying cybersecurity technologies based on projected breach costs.


I. INTRODUCTION
The benefits of renewable energy continue to grow, with wind generation supplying 9.2% generation in the United States (US) [1] but 22.6% in other western countries like Germany [2]. Through diversification and greater distribution system integration, the application of renewable energy The associate editor coordinating the review of this manuscript and approving it for publication was R. K. Saket . promises greater power system resilience from threats that include damaging storms and cyberattack [3], [4]. The ability for communities to meet critical load demand, distribution can lift the resilience burden on transmission systems and large-scale generation suppliers to fulfill these needs. As long as there are no common-mode vulnerabilities, distributed generation promises a reduction in individual threats, because disruptions from compromise are likely smaller in scale and less likely to affect the bulk power system. Looking to the future, distribution and diversification provide practical pathways for dampening climate change impacts and improving resilience.
However, the control systems necessary to integrate renewable energy systems expand the attack surface by exposing more communications interfaces [5]. As a result, cyber-resilience must be elevated to address increasing threat levels to give stakeholders the necessary operational reliability. Advancing a reference architecture that enables secure design across all generation types, large and small scale, is critical to the future of distributed power system resilience.
In the last decade, there has been growing attention to wind cybersecurity and concern for potential impacts to the grid [6], [7], [8], [9]. While demonstrations of the possibilities for attack have been demonstrated by academics based upon local attacks [10], [11] and wind vulerabilities [12], [13], [14], there have also been several attacks on wind turbines, including a cryptojacking attack in [15] and [16], loss of satellite communications to sites [17], ransomware [18] and many others [19], [20]. The U.S. Department of Energy published a roadmap for wind cybersecurity in 2020 that identified several key areas of research. The near term goals of this roadmap include the development of reference architectures for wind sites, best practices to secure wind energy systems, application of intrusion detection, and scalable test beds to show differentiating benefits of cyber-hardening technologies to the industry [21].
One of the leading challenges in cybersecurity is justifying the additional cost to fortify infrastructure and distributing the financial burden of upgrades across stakeholders. There is little research into quantifying the cost-benefit trade space of security tools in operational technology (OT) applications. For this reason, the OT community would benefit significantly from new cyber-resilience metrics that can be used to measure the security improvements from adding additional cybersecurity technologies [22]. Work in this area includes gathering metrics based on characterizing the physical consequence resulting from a physical attack [4], [23], [24], which is critical for OT applications, but also cyber consequences that provide a more generalized understanding of the impact to the communications system [25], [26]. However, an integrated approach is important to providing the actionable perspective for both the cyber defender and power system operator [27] for OT environments.
This project has pursued a new approach to establishing and calculating cyber-resilience metrics for generation assets. In this case, we have created a real-time, high-fidelity power/networking co-simulation cyber range. Previously, Sandia National Laboratories created solar communication networks coupled to investigate cyber-physical interdependencies and options for hardening Previously Sandia National Laboratories has created photovoltaic communications networks [28]. This effort expanded on that work by applying the approach to wind communications and power generation co-simulation environments with different cybersecurity defense and remediation technologies-thereby providing a quantitative environment for evaluating cyber-physical metric improvements with different cybersecurity investments.
Section II provides a background on the test bed with the cybersecurity tools applied on a site network with 30 wind turbine generators (WTGs) in the Electric Reliability Council of Texas (ERCOT) power system. Section III provides a perspective on threat vectors and how the experiments were designed to demonstrate the benefits of individual cybersecurity technologies. Section IV discusses the test results and summary benefits. Last, Section V shares conclusions from this work.

II. VIRTUALIZED CYBER RANGE ENVIRONMENT
A three year effort was devoted to create a high-fidelity network/power co-simulation environment that would accurately represent the real-time impact of adversary actions on the network, computing equipment, and power system. The virtualized environment was created using multiple Sandia National Laboratories-developed Emulytics (emulation + analytics) tools: 1) minimega [29]-a tool for launching and managing virtual machines (VMs) distributed across a compute cluster. 2) phēnix [30]-an orchestration tool allowing easy configuration, access, and interaction with experiments and VMs in minimega. 3) bennu [31]-a tool for simulating control system devices backed by physical system simulators.
These tools work in concert to easily deploy and tear down the cyber range as well as provide researchers access to virtual equipment.

A. COMMUNICATION NETWORK
The communication networks and virtual machines deployed in the cyber range, including cybersecurity hardening applications, remote access for original equipment manufacturers (OEM), grid operators, and owner or operators (O/O), and the wind plant control network are depicted in Fig. 1. These networks and virtual machines were defined and configured using phēnix topology and scenario configuration files [32] and deployed using minimega, which in turn uses QEMU [33] to run the virtual machines and Open vSwitch to create the IP network segments. Once booted, most of the VMs include two network interface cards. One is on the ''experiment'' network representative of a real world configuration and the other is a common management network that does not exist in real world implementations but instead facilitates cyber range interactions between physical system simulators and simulated control system devices. Each VM is provisioned at boot time using scripts injected into the underlying VM image file. The provisioning scripts configure the VM's internet protocol (IP) address(es), start specific services or applications, and configure services or applications specific to the role of the VM in the cyber range.
On the experiment network, there were two sub-networks separated by a wind site firewall. The first network, shown at the top of Fig. 1, represented the internet wide area network (WAN) and included OEM, O/O, and grid operator controllers, a protonuke server that generated fake internet traffic, and an external adversary computer. The second network, shown at the bottom of Fig. 1, were the wind site virtual local area networks (VLANs). There were VLANs for security cameras-represented as protonuke servers, noncritical equipment, for example, a local Wi-Fi network so contractors could browse the internet, and a jump host that would authenticate remote VPN connections before allowing access to the supervisory control and data acquisition (SCADA) VLAN. The SCADA VLAN included the OT/ICS wind equipment along with the cyber-hardening technologies. There was a Windows 7 Wind Site Controller and 30 WTGs represented as SunSpec Modbus-compliant servers. The wind site controller included SunSpec System Validation Platform (SVP) software [34] that was designed to continuously monitor active and reactive power of the turbines, which were pulled from the PowerWorld Dynamics Studio (PWDS) [35] simulation via the common management network every second. The WTG Modbus servers also included active power control points, which were tied to the generator settings in the PWDS model so that writing new setpoints would modify the power simulation.
B. PHYSICAL ARCHITECTURE For the power system emulation, the ACTIVSg2000 synthetic 2000-bus ERCOT model [36], [37], [38] was modified to include 30 WTGs. The WTGs replaced a single wind generator in the original ACTIVSg2000 model on the bus, which mostly impacted the voltage at the point of common coupling when adjusting the turbine output from rated to 0 kW. This model was run in real-time on a separate Windows VM in the cyber range using PWDS and integrated with the bennu PWDS provider to relay measurements and setpoints between other bennu VMs acting as wind turbine controllers over the common management network.

C. CYBERSECURITY TECHNOLOGIES
Secure technologies are necessary to create next-generation, resilient designs for energy applications. To inform a reference architecture design and R&D gaps for the renewables industry, this team conducted a survey to evaluate the current state of the industry [39]. The survey covered multiple cybersecurity tools that could be deployed in wind energy, solar energy, and electric vehicle charging architectures to provide security functions for the owners or operators, as shown in Table 1. The cybersecurity technologies found to be promising to the wind industry included: • Operational technology encryption-This adds confidentiality to OT communications by preventing communications from being intercepted, but at the expense of NIDS deep packet inspection visibility.
• Access control-Security tools that regulate who can view, create, or manipulate resources, which can also be based on a role-based access control (RBAC) proxy.
• Security Information and Event Management (SIEM) systems-A single interface, typically in a Security Operations Center (SOC), with alert information from different security monitoring and detection tools, threat feeds, etc., and potentially integrated analytics for detailed analysis.
• Extended Detection and Response (XDR); Security Orchestration, Automation, and Response (SOAR)-Technologies that ingest NIDS/HIDS data and use automated response tools to defend the system assets with pre-programmed playbook rules. Based on the survey, the team selected or built representative tools for each of these cyber hardening technologies in the cyber range environment, as shown in Fig. 1. 1 The OT Modbus traffic was encrypted using Secure Modbus/Transmission Control Protocol (TCP) [40] implemented in the SunSpec WTG controllers and in a custom RBAC proxy. The proxy authenticated a user via mutual Transport Layer Security (TLS) authentication and queried a lightweight directory access protocol (LDAP) server for access control policies to give users limited, role-based access to read and write actions on the the WTG Modbus server holding registers. Nozomi Networks Guardian [41] was used for the NIDS and Wuzah was used for the HIDS [42]. The SIEM was two Grafana dashboards that pulled data from the Elasticsearch database associated with Wuzah. This project used Palo Alto Networks Cortex XSOAR as the SOAR tool [43]. A bash script was created to scan for NIDS/HIDS alerts in the SIEM and then issue call a XSOAR webhook to trigger the associated playbook.
Several tests were performed to demonstrate the cyberphysical benefits of different cybersecurity technologies. Each test included a different cyber range topology that incorporated these technologies as they would be fielded at a wind site. By phasing in the cybersecurity technologies, the team was better able to quantify the resilience improvements from individual technologies. The five different wind site security topologies are provided in Table 2. The baseline topology only included basic perimeter controls that exist at most wind sites. Encryption and access control were added in the second topology because they provide well-known confidentiality and authentication protections. Topology 3 included a NIDS, which provided sensing of potential malicious behavior on the network and a SIEM that provided analysis and visualization for the cyber defenders. Topology 4 employed the SIEM with the HIDS installed on the jump host and the wind site controller. Finally, in topology 5, a NIDS, HIDS, SIEM and SOAR were integrated with the system. The SOAR enabled the environment to autonomously respond to NIDS and HIDS alerts to block the attacker's IP address.

D. CYBER-PHYSICAL METRICS
Cyber and physical impact metrics were created to correlate security technologies to resilience improvements. The physical impacts were calculated based on the impact on the wind site power production and local voltage measurements. The cyber metric provided a basis for the alignment or threat to the communications infrastructure based on a priori knowledge of the two attacker kill chains [44]-or sequences of adversary actions to reach their objective.
The physical resilience score was the result of two penalty measurements based on the voltage of the system and the active power production of the wind assets. The WTG wind profiles were set to produce full nameplate power (0.8 MW) for the experiments in PWDS. Because there were 30 turbines, the site was anticipated to produce 24.0 MW under normal operations. If a cyberattack reduced production, the generation penalty was represented by, where P nameplate is the production of the wind site and N is the total number of WTGs, 30, at the site. This resulted in a score of 0 when the site was operating as expected and a score of 1 when the site had lost 100% of the generation due to a cyber incident. Notably, with real wind sites, the generation is not fixed at P nameplate , so this term would need to be replaced with P forecast based on local anemometer measurements or some other out-of-band prediction of production. The voltage resilience metric was calculated based on a quadratic penalty function based on the deviation from the nominal voltage (1.0 pu). The voltage metric was calculated using, where α was a scaling factor, V PCC is the voltage of the wind site at the point of common coupling and V nominal is the voltage at the site without any cyber events. When the V PCC was at V nominal = 1.0 pu, the score was 0. The α term was selected to be 100 to bound the result to [0, 1] within a 0.90-1.10 pu voltage deviation. When voltage deviates from nominal voltage, the penalty increases until reaching 1 at ±0.10 pu deviation. Small deviations in are expected in Pen volt under normal operating conditions due to changes in generation and loads. However, lager deviations can be caused by physical degradation of components or loss of components from a cyber or physical event. Therefore, larger deviations from the baseline voltage result in a reduction of the physical score. The physical resilience score, R phys , was determined based on the root sum squared value of Pen gen and Pen volt through  the following relationship, where a score of 1 is good and a score of 0 is bad and means that both the generation has dropped to 0 and voltage is significantly outside of normal ANSI C84.1 Range B limits. The cyber resilience was a measure of the system to resist pre-programmed local and remote cyberattacks. A cyber resilience score of 100 indicated the system has prevented all cyberattack steps for both the local and remote attack sequences, as described in Section III. A resilience score of 0 indicated all attacks were successful and the system did not stop any of the attack steps. During the cyber attacks, the success or failure of the attack was pushed to the centralized Elasticsearch server based on custom response characteristics. The cyber resilience metric on the Grafana dashboard then pulled the number of successful attacks from the Elasticsearch database to calculate the current cyber resilience score, where represents the attacker steps, denoted with remote or local subscripts. successful are the steps that fully executed on the cyber range. There are eight steps in the remote kill chain and five steps in the local kill chain for a total of 13 steps, i.e., the cyber resilience score denominator. Successful execution of each attack step reduces the cyber resilience score by 0.077, so if all attacks are successful the resilience score will go from 1 to near zero.
While this approach worked well for this cyber range, alternative cyber resilience metrics are also possible that do not rely on a prior knowledge of attacker behaviors. These could include control or communication availability of WTGs, level of adversary access to site VLANs or systems, or number of malicious WTG commands/packets. We recommend future work to continue to investigate cyber-physical metrics for wind systems.

E. CYBER-PHYSICAL DASHBOARD DESIGN
A notional dashboard ( Fig. 2) for tracking cyber-physical resilience on a ''single pane'' was created that integrated alerts, trends, and cyber-physical metrics. The diversity of the dashboard ensured a cross-role perspective on how cyber and physical impacts were tied, easily displayed the level of impact, and tracked the overall resilience of the power and communications systems against threats. For this effort, this represented the recognition of the benefits of introducing cybersecurity tools.
In the cyber range, the dashboard utilized Grafana software [45] to visualize and monitor the system by querying the Elasticsearch database. This software lends properties and capabilities similar to most monitoring solutions utilized by OT fields to manage their physical equipment readings. Representing information in a manner that is understood by operators from both specialties allows for easier communication, understanding, and cooperation between these specialties resulting in a more efficient operating environment.
The dashboard was arranged into three distinct sections that utilize common cybersecurity metrics and monitoring to understand the risk and resilience posed to physical assets within the testing environment. The top of the dashboard consisted of metric panels detailing the number of physical log events for different severity levels. This gave operators monitoring the system detailed information about how severe and/or persistent a threat to the system may be. By showing the number of log events, grouped by severity level, over time can allow operators to notice the attacks that may be more calculated or methodical by performing actions over set time intervals. Next is a network diagram presented on the dashboard that associates NIDS and HIDS log events to specific devices within the test environment. The desktop computer icons represent workstations, servers, and OT devices. The icons were color shaded according to the max severity of the NIDS and HIDS log events on a 0-10 range following these rules: Design of a cyber-physical dashboard for wind sites.
Finally, in the bottom pane is a line-graph panel displaying cyber-physical metrics defined in the previous section over a period. The combination of panels displaying event logs grouped by severity and resilience metrics over time gives operators a visual correlation capability to view how different system events may impact the resilience of the testing environment.

III. MODELLING THE THREAT
The threat actors noted in this experiment can be considered having the capabilities of advanced persistent threats. Two attack vectors were considered: one originating from the internet and other via a local OT system, as depicted in Fig. 1. The attack steps are provided for the remote adversary in Table 3 and in Table 4 for the local adversary. Each of the steps of the kill chain was correlated with a Technique from the MITRE adversarial tactics, techniques, and common knowledge (ATT&CK) framework [46] or or ATT&CK for ICS Framework [47]. The fourth column of the tables includes possible NIDS and HIDS alerts that could be generated from the adversary actions. These alerts could then be acted upon by a cyber defender or a SOAR tool, as indicated in the last column of the tables.

A. AUTOMATED TESTING
The team used an INL-developed attack test harness (ATTAR) [50] to automate testing and provide for reproducibility. The ATTAR is a set of small, simple bash scripts designed to run in the virtualized environment for automated testing. The ATTAR included several features: The execution time and room for error were greatly reduced because the attacks were automated via the test harness. In fact, by using the ATTAR tool the entire remote kill chain could be executing in 206 seconds and the local kill chain could be executed in 49 seconds. This was helpful in quickly iterating on red team-blue team scenarios and refining the cyber defense tools as part of the project.

B. TEST HARNESS IMPLEMENTATION
Using the test harness framework, targeted attack steps were scripted for the remote and local kill chains. The remote attacks used the metasploit framework, custom created metasploit modules, and specialized python scripts to deploy a realistic attack on the wind site. In short, the remote ATTAR attack sequence included the following: 1) Establishing the VPN connection to the jump host and starting the metasploit framework remote procedure (startup operation) daemon [48] 2) Nmap scanning to ensure the wind site controller target was online 3) Deploying the EternalBlue payload, which gave remote control of the Windows 7 wind site controller 4) Migrating the meterpreter session into the explorer.exe process to disguise the adversary's presence 5) Exfiltrating password hashes from the site controller's security account manager (SAM) database 6) Adding a backdoor user to the target site controller 7) Adding a scheduled task to the target site controller that would call back to a remote command and control (C2) server in case the Meterpreter session died 8) Creating port forwarding rules on the wind site controller to reach the WTG Modbus servers 9) Sending Modbus writes to change 20 out of 30 windfarm WTGs to 20% active power through the Site Controller 10) Disconnecting from the VPN and shutdown the daemons (cleanup operation) VOLUME 11, 2023  While the first and last items were simply to build up and tear down the connections, as shown in Table 5, the other operations could be theoretically detected by NIDS and HIDS tools that were properly configured (and trained in the case of machine-learning based IDSs). Attack steps that sent traffic over the network, like the Nmap scan and EternalBlue attack, can be detected by NIDS systems that are monitoring network traffic connected to network taps or switched port analyzer/mirror ports. In the case of the operation on endpoints, such as the process migration and creation of scheduled tasks, these can only be detected by HIDS devices on systems with the appropriate host logging functions enabled. Additionally, attacks that impact physical operations, such as the Modbus WTG writes that change site production, can be detected in changes in the power output of the wind farm. This highlights the need to have good cyber-physical data analytics and visualization tools. Additionally, installing diverse physical protection technologies (e.g., safety-instrumented systems, hardened control/fault logic, etc.) can reduce the impact of cyberattacks.
The local network attack sequence targets the wind site like an adversary who has gained local OT network access. This attack kill chain is noisy with network scanning, denial of service attacks, and ultimately reduction of WTG power output. All of the attacks generate plain text network traffic that can easily be ingested by a NIDS except the brute force password attack on the remote desktop protocol (RDP) service. The ATTAR sequence includes the following steps: 1) Perform an Nmap scan on the network to discover hosts 2) Hydra [54] brute force password attack on remote desktop service running on wind site controller 3) Hping3 [55] denial of service attack on wind site controller 4) Hydra brute force password attack on wind site PLC's telnet service 5) Custom python Modbus-based attack, which changes settings on the WTGs to reduce active power output to 20% of nameplate capacity Creating these attack steps took several man-days but the automated execution only takes a few minutes. As a human defender, it would not be possible to react to the automated sequence but in a real cyberattack it is likely the adversary would require several hours or days to complete these steps, in which time a human operator could block or otherwise isolate the adversary.

A. SCORING THE TOPOLOGIES
Five different wind site network topologies were scored in accordance with the cyber and physical resilience metrics described previously. The results of the experiments are summarized in Table 5 and Table 6. Prior to running the local and remote attacks and getting the scores, each environment had the Elasticsearch database cleared by erasing all alerts, power data, and ATTAR data-which were parsed to generate the SIEM dashboard and track kill chain progress. Cyber hardening technology impact on remote kill chain. Red indicates adversary completed step, teal indicates the adversary was detected and an attentive human could potentially block them after this step, and green means the adversary was blocked by the SOAR tool as a result of this step.

TABLE 6.
Cyber hardening technology impact on local kill chain. Red indicates adversary completed step, teal indicates the adversary was detected and an attentive human could potentially block them after this step, and green means the adversary was blocked by the SOAR tool as a result of this step.
In the Topology 1 baseline test, no security protections or orchestration were enabled. As expected, the results represented a worst case scenario, where all adversary attacks were successful. As a result, the remote attack reduced the cyber resilience to 38.5% and the local attack reduced the cyber resilience to 61.5% taking both attacks at the same time VOLUME 11, 2023 reduced the cyber resilience to 0.0%, meaning all attacks were successful. The remote attack took approximately 1 minute to run seven of the eight attacks, gaining complete control of the system; total time to run all eight attacks was approximately 2 minutes.
In Topology 2, access control and Modbus encryption were added and the same sequence of tests was executed. The cyber resilience score increased to 46.2% for the remote attack scenario and 69.2% for the local attack scenario. Because the adversary was unsuccessful in manipulating the WTG power setpoints without the correct TLS credentials, the physical resilience score was 96.0% for both scenarios because the voltage at the wind site point of common coupling is 1.02 pu when the turbines are operating at rated power. However, if the adversary had used their access to the wind site controller to extract certificates, they would have been able to impact the WTG generation and the physical resilience would have dropped to 46.4% and 17.2% for the remote and local attacks-or even worse if the adversary set the turbine active power curtailment to 0% of nameplate.
Topology 3 included the SIEM and NIDS. The NIDS alerted on the reconnaissance, EternalBlue, and WTG Modbus write kill chain steps of the remote attack. If a human actor was closely monitoring the SIEM, they may be able to block the attacker before Step 3 was executed. This would have increased R cyber from 38.5% to 84.6% and R phys from 46.0% to 96.0%. Similarly, the NIDS picked up the reconnaissance and WTG Modbus write kill chain steps of the local attack. Again if a human cyber defender was quick, they could have isolated the attack before Step 2 and increased R cyber from 61.5% to 92.3% and R phys from 17.2% to 96.0%.
In Topology 4, encryption, SIEM, and HIDS cybersecurity defensive tools were incorporated. Without human intervention, there was no difference between the cyber resilience score from Topology 2. The encryption again blocks all attempts to reduce the resilience of the wind turbine network just as in Topology 2. In this topology, the HIDS alone does not stop or block the attack, but if a human actor investigated the HIDS alerts they would have been able to stop the kill chain sequence before Step 7 of the remote attack and before Step 3 of the local attack-bumping R cyber from 46.2% to 53.8% for the remote attack and 61.5% to 84.6% for the local attack.
Last, Topology 5 included the NIDS, HIDS, SOAR and SIEM. In the event of a NIDS alert with a severity of ≥9, the script would trigger the remote attacker defense playbook, which would confirm that ''EternalBlue'' was in the NIDS message with a call to the Elasticsearch. The playbook would then continue and take the source IP from the NIDS message and Secure Shell (SSH) into the wind site firewall to restrict access to the SCADA VLAN by blocking the VPN IP address to the wind network. The second orchestration playbook was designed to handle a local brute force attack. If there was an alert from the HIDS with a severity >5, the script would call the local attacker defense script that would confirm multiple ''failed logins'' messages with the Windows 7 wind site controller. At that point the playbook would find the offending source MAC address and SSH into the OT switch to disable the switch port the MAC address was connected to (simulated with a change in the Open vSwitch configuration).
This configuration resulted in a significant increase in cyber resilience because of the automatic reaction of the playbooks. In the case of the remote attack, the EternalBlue attack succeeds, but then the attacker is blocked from the network. For the local attack, the RDP brute force attack is terminated before it can finish. To calculate response times for the SOAR playbooks, each attack was run five times. The EternalBlue SOAR playbook executed in 2, 3, 8, 8, and 8 sec (5.8 sec average) and the local attack RDP playbook completed in 11, 8, 18, 17, 15 sec (13.8 sec average). The difference is because the RDP playbook looked for eight failed logins before triggering the playbook. As a result of the SOAR response, the cyber resilience score is 76.9% for remote attack because EternalBlue completes and 84.6% for local attack because the RDP brute force is prevented by the SOAR tool. The physical impacts are prevented to produce a score of 96.0%. If a human were to respond to the reconnaissance alerts, the cyber resilience score could be increased by 7.7% for the remote and local attacks.
The signature-based SOAR implementation produced a zero false-positive rate for the experimentation, however this is a specially tailored defensive response based on known alert messages. In a field installation, the playbooks would not be fully tuned to the HIDS and NIDS alerts. Instead, it is more likely a human would need to take appropriate response actions.

B. SIEM DASHBOARDS
As described earlier and shown in Fig. 2, the SIEM dashboards were designed to provide insights into what wind site cyber-physical data a cybersecurity defender in a Security Operations Center would see. Using data from the Elasticsearch database, a representation of the cyber and physical scores were created as shown in Fig. 3. This dashboard was captured after the Topology 3 remote attack, wherein the power was dropped on 20 out of 30 WTGs. This changed the site production from 24.0 MW to 11.2 MW (or 0.373 MW/turbine), which is shown in the power level. The voltage also shifts from slightly over voltage (1.020 pu) to slightly under voltage (0.979 pu) as each of the WTGs have their active power limits reduced through the Modbus holding register write operations. In response to this attack, the cyber and physical resilience scores were updated, as shown in the lower portion of Fig. 3. These values match the remote attack for Topology 3 with R phys = 46.4 and R cyber = 38.5.
The dashboard shown in Fig. 4(a) shows a collection of NIDS alerts after the local attack. The ''TCP SYN flood'' is triggered by the Nmap reconnaissance, the ''Protocol-based flood'' is triggered by the DoS attack, and the ''Cleartext password'' alert is generated by the Telnet brute force attack. Nozomi also produces a ''New global function code'' alert when it detects a new Modbus function code 16 (Write Multiple Registers) on the network. However, subsequent attacks do not alert on this event, presumably because the NIDS has learned this is normal behavior. This shows the importance of detecting the attack early. The ''Invalid IP'' is not related to the attack and was either a false positive or other networking noise in the system. Fig. 4(b) shows NIDS alerts of a given severity versus time. For the time frame shown there are two Severity 9 alerts, related to the EternalBlue attack, and one severity 7 alert, related to the Nmap scan. Fig. 4(c) shows a network map with color coding associated with alert severity. This screenshot was taken immediately after the local attack Modbus writes, where ''New global function code'' and the ''New global variable producer'' Severity 5 NIDS alerts, related to the new Modbus traffic, were detected by the NIDS. Based on the destination IP addresses, the dashboard showed that all WTGs were targeted in this attack. As described earlier, we found that Nozomi did not consistently alert on Modbus events, especially if this traffic had been seen previously.

A. STUDY LIMITATIONS
As shown in the cyber and physical metrics in Tables 5 and 6, there are clear beneficial trends when including security tech-nologies. But the benefit of the cybersecurity technologies depend on the type of attack vector and adversary tactics, techniques, and procedures (TTPs). For instance, TLS encryption protected the OT network traffic from being intercepted by malicious intruders and TLS mutual authentication foiled attempts to write to Modbus holding registers in the WTGs. In comparison, the RBAC implementation did not increase the cyber or resilience metrics because the adversary did not incorporate the Modbus proxy as part of the kill chains by, for example, using compromised TLS certificates from the OEM, O/O, or grid operator. For Topology 2 and 4, based on the level of access by the adversary, additional steps could have been taken to bypass the encryption and access control protections. The NIDS and HIDS worked well to detect abnormal traffic and host actions but there was noise in the signal. Some of the false positives-defined as alerts that were unrelated to the cyberattacks-included HIDS alerts for normal user logons and logoffs and NIDS alerts for packets with the same IP source and destination. Examples of false positives alerts are outlined in Table 7.
After careful construction, the SIEM dashboard provided cyber defenders awareness of appropriate mitigative actions. Configuring these displays would be challenging for field installations because only critical information should be displayed in the SOC. Depending on the wind site design,  there could be substantial differences in the layout and data presented on the SIEM dashboards. Also, depending on the cybersecurity team, different information or displays may be preferred.
A SOAR tool introduces automated playbooks that provide a step-by-step flow of actions to expediting response. However, playbooks take hours to create and test, and playbooks must be created for each type of suspected attacks. These playbooks would not provide any security improvements for zero day vulnerabilities or attacks with unknown signatures because the NIDS or HIDS would not detect these attacks.

B. CONFIGURING CYBER HARDENING TECHNOLOGIES
The cyber resilience of the wind site is based on its ability to prevent, recognize, and mitigate cyberattacks. Each of the security technologies provide features to more effectively defend the wind system but only when configured correctly. Operational environments are likely to require further refinement. The following are a set of considerations when configuring and relying on these technologies in ICS networks.

1) ENCRYPTION
Encryption was provided using Secure Modbus/TCP [40] for the SunSpec protocol. This functionality was added to pySun-Spec [56] to provide confidentiality in the SunSpec Modbus communications. One of the downsides of encrypting the Modbus (or any other OT traffic) is the inability for deep packet inspection tools to work on these connections. That is, if there is a malicious command or other message sent over an encrypted channel, it is not possible for NIDS tools to detect that operation and send an alert.

2) ROLE-BASED ACCESS CONTROL
The RBAC was implemented in a custom Modbus proxy that authenticated users via mutual TLS authentication and queried an LDAP server for access control policies based on the role extension present in the Modbus client TLS certificate. The role-based access control policies were enforced at the proxy and would permit Modbus WTG read/write actions for the user based on their role. Alternatively, less granular approaches to this implementation include restricting access to the OT VLAN based on roles or ''all-or-nothing'' access to the WTG controllers, in turn, based on a role present in the certificate.

3) NETWORK-BASED INTRUSION DETECTION SYSTEM
Nozomi Networks Guardian requires a training period to learn normal network patterns. This training data set needs to include a full range of operations or there is the risk of false positives once it is switched into protecting mode. For the wind simulations, the NIDS was trained with 1 hour of network data that included normal wind site controller interactions with the WTGs. If other actions are likely in the environment, such as remote access by owners or OEMs, this traffic should be included in the training data set. While that initial training set was used to generate alarms, the system continues to learn. Therefore, if an adversary is interacting with the system over an extended period, the likelihood of detecting their actions decreases over that period because the machine learning NIDS tool begins to recognize this traffic as normal.

4) HOST-BASED INTRUSION DETECTION SYSTEM
HIDS endpoint tools, like Wuzah agents, forward local logs to a centralized server. It is essential that appropriate logging is enabled on the endpoints so that HIDS tools can detect adversary actions. For instance, the Wuzah agent was not able to detect the creation of a new user or brute force attack on the Windows 7 wind site controller until system audit policies were modified. Critical systems should be carefully configured to ensure appropriate logging information is available. It is also helpful to perform penetration testing of these systems to validate they are configured and operating as intended.

5) SECURITY INFORMATION AND EVENT MANAGEMENT
The SIEM system is designed to provide a single location to visualize corporate-wide threats. This means that critical logs and alerts need to be captured by the associated database and easy-to-use tools need to be created to visualize and interact with this data. SIEM industry leaders have mechanisms to quickly pivot though massive data sets to perform threat hunting operations. In the case of the wind systems, not all the IDS and SOAR tools had easy Elasticsearch integrations, so custom code was created to push data and pull data from the database.

6) SOAR IMPLEMENTATION
The XSOAR playbooks were created using drag and drop graphical programming methods to create flow diagrams. There were two playbooks created to stop attacks. One blocked the remote adversary after detecting EternalBlue on the network and the other blocked the local attacker after detecting the RDP brute force logons. Normally, NIDS and HIDS tools would be configured to trigger SOAR playbooks, but these capabilities were not possible with the tools used in this project. To get around this barrier, we created a playbook trigger script to poll the Elasticsearch database looking for specific messages in the alerts and call the SOAR playbooks via webhooks. A better solution for triggering SOAR playbooks would be creating direct connections within Nozomi and Wuzah to directly trigger SOAR playbooks with webhooks or using RabbitMQ [57] or Kafka [58] to establish a pipeline for analyzing the severity of the alert and to determine if it rises to the level of direct, immediate action from the SOAR system.
We found that the learning curve for the SOAR tools was steep. Incorporating these technologies into wind systems will require substantial time for highly qualified security defenders. However, once configured correctly, the SOAR tool quickly and accurately isolated the adversary and prevented physical impacts to the system.

C. BENEFITS AND VALUE PROPOSITION FOR INDUSTRY
Based on discussions with industry, the value proposition for the engineers and scientists responsible for the secure, reliable operation of the wind farm is different than that of the chief information officer and other ''C'' suite executives. In particular, the engineers and scientists require a technical correlation of the benefits to ensure the security and reliability. However, the C suite will require correlation of the benefits in terms of savings from cybersecurity insurance expenses, probability of losses, etc. Table 8 includes rough estimates on a cost basis for adding security with a perspective similar to a physical investment. For example, this table establishes a baseline to compare upgrading the wind turbine generators compared to adding new security measures-where the former adds generation capacity but the latter protects owner assets.
Within Table 8, the estimated technology costs and labor for commercial technologies is based on team experience and information gathered for different installations. Estimates VOLUME 11, 2023  are provided as a range because the cost can vary widely dependent on the technology chosen for the need, the capabilities of the technology, and the size of the site. Labor will vary dependent on the size of the site, network traffic, reporting requirements, etc. The cost of the breach is based on a 2022 IBM report that evaluated the cost [59]. Here we make an assumption that there will be a wind site breach once in a year, noting the significance that a single event will come with an average cost of $4.82M. If better breach wind site probabilities are available, this value can be adjusted to recalculate payback periods and determine if cybersecurity investment is financially prudent. This analysis shows there is a return on investment for the cyber hardening technologies in less than one year if there is at least one cyber breach in that same period.

VI. CONCLUSION
The application of cybersecurity technologies have become commonplace in many sectors. Yet, critical infrastructure comes with unique trade-offs because cybersecurity investments are difficult to quantify, because there are few cyber-resilience metrics or data on costs from cybersecurity threats. The outcomes provided in this R&D project have helped quantify benefits in terms of risk avoidance when making investments in cybersecurity technologies. The results depend significantly on the attacker TTPs, however; if the attack methodology changes, so too will the cyber and physical metric improvements. Running several cybersecurity kill chains in a virtualized environment can illustrate a diverse mitigative benefit to a wide spectrum of attack vectors, and this will point toward a comprehensive solution that tips the cost-benefit scale toward cybersecurity tool and training investment. In this work, we found a quantitative increase in cyber and/or physical resilience metrics when incorporating OT encryption, intrusion detection systems, monitoring tools, and response orchestration technologies in a virtualized wind site attacked by a local and a remote adversary. Based on a cost-benefit analysis, it was found wind owners, operators, OEMs, and grid operators are financially justified in adopting these technologies to reduce the likelihood of a damaging attack on wind assets. Furthermore, the reference topologies and Elasticsearch data have been open-sourced [60] to enable others to explore different security technologies and metrics-based benefit comparisons. The team encourages research entities to continue to inform the value proposition of adding security systems to renewable energy assets. Follow-on research from this project should include development or refinement of new cyber-physical resilience metrics; conducting red team/blue team scenarios with the wind site cyber range to determine the human response characteristics; and incorporating higher-fidelity turbine models and wind site controllers that are more representative of those in the field. He is currently a Distinguished Member of Technical Staff with the Sandia National Laboratories. He leads several multidisciplinary renewable energy research projects focused on power system interoperability, control, optimization, and cybersecurity. He is also the Co-Convenor of the SunSpec/Sandia Distributed Energy Resource (DER) Cybersecurity Workgroup and investigating cyber-hardening technologies for photovoltaic systems, electric vehicle chargers, wind energy sites, and microgrids.
BRYAN RICHARDSON is a cybersecurity researcher and a practitioner currently supporting multiple public and private customers on red team, threat hunting, network security design, and software development activities for both operational technology and traditional enterprise environments. He formerly served as the Chief Technology Officer of Dark Cubed for almost five years, where he architected and built the Dark Cubed platform from scratch. Before that, he served for over ten years at the Sandia National Laboratories performing extensive research, development, and engineering in cyber security for industrial control systems. He also supported the National Cybersecurity Programs at the Department of Homeland Security for several years after leaving Sandia.
CRAIG RIEGER (Senior Member, IEEE) received the B.S. and M.S. degrees in chemical engineering from Montana State University, in 1983 and 1985, respectively, the P.E. degree from Idaho State University, and the Ph.D. degree in engineering and applied science from Idaho State University, in 2008. His Ph.D. coursework and dissertation focused on measurements and control, with specific application to intelligent, supervisory ventilation controls for critical infrastructure. He recently retired as the Chief Control Systems Research Engineer and the Directorate Fellow with the Idaho National Laboratory (INL), pioneering interdisciplinary research in next generation resilient control systems. The grand challenge provided an integrated research strategy to address the cognitive, cyber-physical challenges of complex control systems into selfaware, trust-confirming, and threat-resilient architectures. In addition, he has organized and chaired 14 co-sponsored symposia, one National Science Foundation Workshop in this new research area, and authored more than 75 peer-reviewed publications. He has 20 years of software and hardware design experience for process control system upgrades and new installations. He has also been a supervisor and technical lead for control systems engineering groups having design, configuration management, and cybersecurity responsibilities for several INL nuclear facilities and various control system architectures.
RAFER COOLEY (Member, IEEE) is currently pursuing the Ph.D. degree with the University of Wyoming Cybersecurity Education and Research Laboratory. He is currently an Intern with the Idaho National Laboratory. His research interest includes bio-inspired evolutionary algorithms called artificial immune systems to develop malware detection techniques. The goal of his research is to be able to detect similar code and behaviors of malware in a scalable manner. He has experience in incident response, digital forensics, security orchestration, automation, and response. He is also an Adjuct Professor with the College of Eastern Idaho. His research interests include CISSP, CEH, and Cloud+.
TYLER PHILLIPS (Member, IEEE) received the B.S. and M.S. degrees in mechanical engineering from Boise State University, in 2009 and 2014, respectively, and the Ph.D. degree in computing, computational math, science, and engineering from Boise State University, in 2020. He is currently a Postdoctoral Researcher with the Idaho National Laboratory, Energy and Environmental Science and Technology Department. He has been an Active Institute of Electrical and Electronics Engineers Member, since 2015, and its affiliated Power and Energy Society. His research interests include power and energy system simulations, numerical computing, and data analysis in the areas of resilient control, resilience metrics, integration of renewable generation, microgrids, and dynamic line ratings. He is also involved with the interaction of human factors in power system.
BEVERLY NOVAK received the B.S. degree in computer science from Montana State University, in 1989, and the M.S. degree in computer science from the University of Idaho, in 2012. She is currently a Software Engineer with the Idaho National Laboratory. She has experience in SIEM and IDS Systems. She was the Principal Researcher of the Quality Assurance for Software Analysis and Resilience Project. She has been on many different projects at INL including research on the More Situational Awareness for Industrial Control Systems (MOSAICS) Project, customer lead for the Wireless Testbed, and the Database Manager for the TRIPS Project.
MEGAN CULLER (Member, IEEE) received the B.S. degree in electrical engineering from Texas A&M University, in 2019, and the M.S. degree in electrical engineering from the University of Illinois at Urbana-Champaign, in 2021, concentrating in both degrees on power systems and electrical engineering. She is currently a Power Engineer and a Researcher with the Idaho National Laboratory, where she contributes to several projects in the infrastructure security domain. She has experience in cybersecurity and resilience for power systems, with a focus on distributed energy resources. She promotes a risk-based approach to both resilience and cybersecurity, investigating these issues for wind energy, solar energy, storage technology, and grid-enhancing technologies. She has been the Co-Chaired the IEEE Power and Energy Conference at Illinois.
BRIAN WRIGHT received the B.S. degree in electrical and computer engineering from the Missouri University of Science and Technology, in 2009, and the M.S. degree in electrical and computer engineering from the University of Illinois Urbana-Champaign, in 2013. He is currently a Principle Member of the Technical Staff with the Sandia National Laboratories. He performs operational technology cybersecurity research, red team assessments, and large scale cyber modeling and simulation research. VOLUME 11, 2023