Runtime Orchestration of Distributed Control System Services with TOSCA, Kubernetes, and GitOps

Runtime orchestration of cloud-native microservice systems remains a challenge with many technology options, such as Kubernetes for managing containers and OASIS TOSCA, a standard for orchestrating cloud services. Researchers and practitioners have proposed different approaches on how the benefits of Kubernetes and TOSCA could be combined, but followed different design philosophies and did not reach consensus. An analysis of the different combinations is missing and existing TOSCA and Kubernetes combinations do not consider the integration of real-time embedded controllers without container workloads, nor a safeguarding of configuration changes, which can be achieved using the GitOps paradigm. In this experience report, we describe and analyze different options for combining TOSCA and Kubernetes. Following one of the options, we have enhanced an existing TOSCA orchestrator to demonstrate software updates across heterogeneous software services as well as a GitOps extension for TOSCA service models. We found that TOSCA workflows are well suited for updates spanning different technologies and that a custom GitOps operator is needed in case not all components are orchestrated by Kubernetes.