Availability Modeling and Sensitivity Analysis of CELERY Airflow Service in a Node-Based Computing Environment

Apache Airflow is an open-source platform to programmatically author, schedule, and monitor workflows. Work-flow management systems (WFMSs) are critical in today’s business landscape. Their main aim is to streamline and automate the execution of various business processes, ensuring that large enterprises can operate with maximum efficiency. This paper proposes availability models utilizing hierarchical modeling techniques to assess the CELERY Airflow service within a node computer system context. In our baseline scenario, we configured the environment with a single computational node, a storage system, and the CELERY service, comprising Web Service, Workers, and Schedule components. We also use Sensitivity Analysis technology to investigate bottlenecks and propose improvements and solutions for planning these systems. The results of the presented case studies show that the system’s downtime can reach almost 100 hours. The sensitivity analysis shows that the system may be improved if the administrators apply some techniques in the application layer.