Pipekit: A Deployment Tool with Advanced Scheduling and Inter-Service Communication for Multi-Tier Applications

Modern cloud applications are based on microservice architectures. The deployment of these microservice based applications often requires that every constituent service starts after all its dependencies are configured and running properly. It is also common that these dependencies generate dynamic data that needs to be supplied to other services too at starting time. More complex scenarios require additionally interchanging data in other phases of the microservices lifecycle. One alternative to solve these dependencies is to describe the deployment of microservice applications manually—using scripts—which allows IT operators to precisely define when a service is ready to start serving other components. However, synchronization by scripting is tedious, error prone and hard to maintain. Other solutions offer specific languages to describe service dependencies, along with tool support that interpret scripts in these languages to take care of starting services in the proper order. These tools are either very rich but complex to use, or fail in providing sophisticated ways to describe what it means for a service to be ready. Moreover, the communication layer between services, if supplied, is based on intermediate entities and non-trivial network protocols. This paper proposes pipekit as a solution, by offering a container orchestration language which focuses on simplicity (pipekit is similar to Docker Compose) and is equipped with directives to define when a service is ready. The pipekit tool provides a communication layer for moving data between services, implemented using shared storage. This shared storage provides a very simple interface to move artifacts between services, and greatly simplifies the synchronization logic of pipekit by using semaphores at the file system level.