Recent Advances in Data Engineering for Networking

This tutorial paper examines recent advances in data engineering, focusing on aspects of network management and orchestration. We provide a comprehensive analysis of standardization efforts as well as platform development activities related to data engineering driven network design. We then focus on the integration aspects of the data engineering ecosystem and telecommunication networks. The results of our tutorial investigation show that despite various efforts towards standardization and network management and orchestration platforms, there is still a significant gap in applying recent developments in the evolving data engineering world to the telecommunication domain. New advanced functionalities in data engineering as well as clear separations between the building blocks of data engineering pipelines within the proposed standardized architectures have been overlooked or not explored in detail by the standardization or platform development bodies in the telecommunication domain. Therefore, at the end of the paper, we discuss these gaps and research challenges in the context of future development processes for data engineering-driven network design and applications of data engineering concepts in telecommunication networks. We also propose several recommendations for early adoption of these technologies and frameworks in telecommunication infrastructures and platforms.