How Degrading Network Conditions Influence Machine Learning End Systems Performance?

As intelligent knowledge-based decision makers, Machine Learning (ML) end applications highly depend on input data quality. Systems that integrate ML-end applications use computer networks for data delivery from the data source to ML processors. Packet losses and lack of resources are the factors that may result in a quality degradation of data transmitted over computer networks, and consequently negatively impact the accuracy of ML applications. In this paper, we investigate the relation between network quality of service (QoS) degradation and ML-end decision making performance based on the data transmitted over various network conditions. For realistic testing, we implement an empirical study, in which we leverage the POWDER platform that provides real-world wireless 5G network testbed. In our experiments, we transfer media-files between the nodes of this real network under various QoS conditions. We investigate the effect of degraded data quality due to those network disruptions on ML-based systems used in different domains. Specifically, we investigate the accuracy of different image classification and speech transcription models in scenarios with higher packet loss in the communication channel and resource shortage on the receiving end. Our results show that network conditions stability is critical to avoid ML misclassifications, and that data quality degradation affects speech transcription model performance to a higher degree compared to considered image classifiers.