Automated Testing and Virtual Commissioning of Automation Networks

Automation networks are a core component of the mission-critical infrastructure in a distributed control system (DeS) as they connect all equipment and services of a DeS. Today, DeS network testing is typically done indirectly by evaluating the primary automation and system functions. However, the convergence of information (IT) and operation technology (OT) networking and the adoption of IP-based protocols have led to increasingly complex software configurations and cyber security risks. These developments demand more thorough and explicit testing of the infrastructure, akin to the testing conducted for applications. Performing this testing is traditionally limited as not all parts of the DeS are available during factory acceptance testing (FAT). In this study, we describe the requirements and solution approach for the systematic and automated testing of converged automation networks. This approach has three key components: re-using communication intents from automation engineering to generate comprehensive test cases, auto-configuring virtualized packet manipulation tools to act as traffic generators and analyzers and executing these tests on segments of a real or emulated network. We then demonstrate the feasibility of this approach using the Mininet network emulator in conjunction with the Scapy packet manipulation tool. While this method may be applied to cover all network logic, it is inherently limited in addressing performance aspects such as bandwidth guarantees, latency bounds, or failover speed. Additionally, we find that we cannot quantitatively measure the fidelity of our testing approaches. Based on these findings we finally identify concrete gaps in the state of the art of automated network testing, proposing concrete activities toward a formal theory of network testing and a standardized/open testing framework.