A Scalable Design Approach for State Propagation in Serverless Workflow

Serverless development is challenging as applications are composed of stateless and short-lived functions. Many workflows require time-bound functions to transfer their state to other function before termination. The serverless Function-as-a-Service offerings lack state management support; therefore, it must be handled at application-level. In this paper, we propose a scalable design approach that simplifies development of workflows that require sharing of ephemeral intermediate data. Our design uses object serialization/deserialization with cloud object storage to share state across functions. It provides a mechanism for fine-grained support for state propagation and synchronization in a serverless workflow. This solution is cost-effective and efficient as it does not depend on any external database or cache for state management. The design has been validated by implementing ‘Word Count’- a classic MapReduce use case. Our results show that the proposed scalable design can process input of any size and can handle state propagation in complex serverless workflow.