A Data Cleaning Framework for Water Quality Based on NLDIW-PSO Based Optimal SVR

Water quality monitoring is an essential part of water big data analysis. Spatiotemporal variations of water quality and constraints on measurement make it very complex. The objective of this study is to establish a water quality data cleaning framework based on time series, in order to clean the water quality data of the Gaobeidian Sewage Treatment Plant inlet in Beijing. Pauta criterion was used to deal with single water quality indicator. For abnormal values and missing values that are discontinuously distributed over time, the average of the non-abnormal data for three days before and after was used to fill it; For abnormal values and missing values that are continuously distributed over time, using the Non-Linear decreasing inertia weight particle swarm algorithm (NLDIW-PSO) based optimal Support Vector Regression (SVR) to forecast. And Pearson's correlation coefficient was used to reduce the dimension of the inputs of the model, k-fold cross validation was also used to train the model. The performance of the model was evaluated in terms of the coefficient of determination (R2), Pearson's correlation coefficient. Water quality data of Gaobeidian wastewater treatment inlet in Beijing, China was taken as the study case to examine effectiveness of this approach. The experiment results also revealed that the proposed model has advantages of stability and time reduction in comparison with other data-driven models including traditional BP ANN, Bayesian network model and Decision Tree model. And this framework can be used as an effective approach to deal with General time series data.