Evaluating Machine Learning Methods for Outlier and Missing Data Handling in Water Supply Network

The digital transformation of water supply infrastructure necessitates effective management of the water flow data transmitted from field devices to servers. However, this data transmission often introduces noise, which negatively impacts data usability and the management of water supply networks. This research investigates and compares various machine learning techniques for processing the noisy water flow data collected by Ben Thanh Water Supply Joint Stock Company in Ho Chi Minh City. The study addresses the challenges of outlier detection and missing data imputation in flow rate datasets spanning from January 2022 to June 2024. For outlier removal, the IQR-Tukey, Z-score, DBSCAN, and Hampel Filter methods were applied and evaluated visually using correlation graphs. To handle missing data, the Simple Imputer, Interpolation, Iterative Imputer, and KNN Imputer techniques were implemented, and their performance was assessed using Root Mean Squared Error (RMSE). The findings indicate that the IQR method demonstrates the most stable and effective performance for outlier removal, preserving the underlying data characteristics. Furthermore, the KNN imputation method exhibits superior accuracy in handling missing data compared to the other evaluated techniques based on MSE scores. This research provides a valuable comparative analysis, contributing a foundation for selecting appropriate machine learning methodologies to enhance the quality and usability of water flow data in water supply companies, thereby supporting standardized data management and improved analytical capabilities for strategic planning.