Benesh: a Framework for Choreographic Coordination of In Situ Workflows

The growing scale of high-performance computing systems increasingly enables scientists to develop more complex applications as in situ workflows composed of coupled simulation and analysis codes. It is therefore important that workflow programming systems and runtime middleware support the composition and execution of these complex applications intuitively and efficiently. The scientific computing community has put significant effort into purpose-built coupled simulation codes that have been optimized for specialized use cases. However, the development effort involving the coupling of established codes has been largely ad hoc. The Benesh programming system was recently proposed to support the development of coupled simulation workflows from existing code bases. Benesh allows a shared data model to be defined across established codes, so that they can be interfaced in a flexible, coupled workflow. In this paper, we develop Benesh into a workflow development framework. Using Benesh, we develop workflow data model and data exchange definitions for coupled, in situ workflows. We evaluate the cost of development using Benesh in terms of development time and overhead, showing that Benesh offers development advantages without undue impact upon workflow performance.