How Can We Improve Data Quality for Machine Learning? A Visual Analytics System using Data and Process-driven Strategies

ML (Machine learning) models are used to mine inconspicuous information in big data. The model and data quality influence the performance of a machine-learning model. However, it is inefficient to modify the model, which is a black box, and low-quality data tends to cause biased learning of the model. Therefore, it is crucial to improve the data quality. Different techniques have been used to improve data quality depending on the data conditions and the data quality issues. Therefore, improving data quality is time-consuming and challenging for users with insufficient knowledge of data. Visual analytics techniques have been proposed to focus on decision support to improve data quality. However, existing studies are complicated for users to consider a comprehensive DQI (Data Quality Improvement) method for generating data suitable for ML models. Also, it remains limited in that users must directly consider all combinations of DQI processes. This paper presents a novel visual analytics system that manages data quality for use in ML models. The proposed system suggests an optimal quality improvement process with visualization techniques such as heatmap, histogram, and scatter plot to support DQI.