Data Perspectives in AI-Assisted Fiber-Optic Communication Networks

Machine learning (ML) is widely being regarded as a powerful technology to solve various complex problems in fiber-optic communication networks. Recently, ML algorithms have been considered for a variety of use-cases in optical networks ranging from the design of network devices to the compensation of transmission impairments to predicting network-wide traffic flow patterns. As ML techniques are inherently data driven, the availability of relevant data sets largely dictates the performance of developed ML models. Presently, majority of ML-aided methods in the literature rely either on synthetic data from certain simulation tools or exploit data from some simple laboratory setups. However, developing robust ML-based solutions for real fiber-optic networks demands assembling big data from various parts as well as across different layers and domains of the network. Unfortunately, there is currently a lack of understanding among the research community about various crucial data aspects in practical optical networks as well as key challenges faced. While there has been a plethora of works discussing numerous ML applications in fiber-optic communications and networking, to the best of our knowledge, there is presently no paper which particularly focuses on data perspectives. In this paper, we aim to shed light on various critical data factors by first describing the tools and technologies imperative for adequate network monitoring and data acquisition. We also elucidate different data types, their physical sources, and their applications. Several useful data handling and pre-processing techniques for efficient network data collection, storage, presentation, analysis, etc., are discussed. Major challenges in the generation, transmission, and sharing of the data are highlighted and some potential solutions to the outstanding problems are proposed, thus paving the way for seamless availability of pertinent data sets essential for realizing intelligent data-driven operations in future fiber-optic networks.