BBC: A DSL for designing cloud-based heterogeneous bigdata pipelines

Several frameworks are available for processing and analyzing the bigdata. But there is limited support available in integrating such heterogeneous bigdata programs to create a bigdata pipeline. In this paper, we introduce BBC, a Domain Specific Language (DSL) designed to create a bigdata pipeline from the heterogeneous programs. The paper focuses on Spark jobs but using a similar approach the DSL can be extended for other frameworks. We have successfully used BBC DSL to design the The Home Depot (THD) search evaluation pipeline that is instrumental in deploying every search feature to the THD website.