Logical Optimisation and Cost Modelling of Stream-Processing Programs Written in a Purely-Functional Framework

We present a vision for the automatic optimisation of distributed stream processing programs. StrIoT -a distributed stream-processing framework built using purely-functional programming – enables a set of validated logical optimisation rules to generate a set of possible deployment plans. A cost model then filters and ranks the plans before the best is automatically deployed across the cloud and edge devices. We describe StrIoT’s functional operators for writing stream-processing programs; the design, implementation and performance of StrIoT’s logical optimiser; and the cost model, which filters and ranks re-written programs and deployment plans in terms of two non-functional requirements: bandwidth and cost. The StrIoT vision is being explored through an open-source proof-of-concept implementation. We present our initial results with a motivating example before outlining the success criteria for future work in this area.