A power fusion data cleaning method based on exponential moving average and cosine similarity algorithms

With the rapid development of the distribution Internet of Things, massive heterogeneous data is continuously generated from the production, transmission and consumption end. Power big data is the key basis to support the business analysis of smart grid. However, the current power big data has the characteristics of fast update speed, high data complexity, low value density and strong time series. This has higher requirements on the speed of data processing and the guarantee of data quality. To solve the above problems, this paper proposes a power fusion data cleaning method based on exponential moving average and cosine similarity algorithm. The exponential moving average algorithm uses a sliding time window to repair and complete missing data, which greatly improves the data repair speed under the power data discontinuous missing mode. The cosine similarity algorithm comprehensively considers the power consumption law of the same attribute users, and can ensure the data repair accuracy in the continuous missing mode of power data. Finally, the test results on real data sets show that the fusion of the two methods can repair large-scale power default data more quickly and efficiently, and greatly improve the robustness of the cleaning method to adapt to different power data missing scenarios.