Power Data Quality Assessment based on Entropy Weight Method and Grey System Theory

With the increase in the scale of power systems, the development of measurement technology, and the decrease in costs, the amount of data in power systems is showing a rapid growth trend, gradually acquiring the characteristics of big data. Making full use of big data to improve the planning, operation, and control of power systems has received increasing attention. How to evaluate the quality of big data is an important issue worthy of study. There have been considerable research reports on data quality improvement techniques such as data cleaning, data integration, and similar record detection. However, research on data quality evaluation is still in its infancy. Against this background, a comprehensive evaluation method for the quality of power big data is proposed based on the characteristics of power systems and the quality characteristics of power big data. Firstly, an index system for evaluating the quality of power big data is constructed. Then, for the time-sensitive problem of big data processing, a MapReduce parallel K-means clustering algorithm is used to achieve fast preprocessing of big data sample sets. Afterwards, the objective weights of various data sets are calculated using entropy weight method, and the gray evaluation method is used to determine the level of data quality. On this basis, a comprehensive evaluation of sample data sets is realized.