Lineage Chain Mark Fault-Tolerant Method for Micro-Batching Monitoring Data in Distribution Power Network

Aiming at the problem of lacking efficient distributed fault tolerant mechanism for data explosion in the distributed distribution power automation system, based on the record-level fault tolerance of a resilient distributed dataset (RDD) and micro-batch computing, a new lineage chain mark (LCM) fault-tolerant method is proposed. Using the distribution power network monitoring dataset as cache object, the programming model based on the micro-batch processing framework is comprehensively used to design the topology example of the distribution power network monitoring data, which can shorten the derivative chain of RDD. This paper takes the monitoring data in a dispatching and monitoring system of the Shijiazhuang-Dezhou distribution power network as the data source, which has deployed a flow calculation test cluster of one primary node and three working nodes. Taking the most frequent fault of single data node failure condition as an example, the test results based on 3 million monitoring data in RDD show that, the CPU usage volatility of this model in cluster nodes was less than 1.5%, the network consumption of read and write execution is decreased by about 3.1% and 5.0%, respectively, compared with the original model, and the disk usage was reduced by 4.2%; the iterative test shows that the number of iterations is positively correlated with the computational time, which is consistent with the calculation formula for the worst execution time of the lineage chain labeling model. When the number of iterations is more than 400, the greater the number of iterations, the more significant the LCM method improves the performance of the calculation. The results show that the long pedigree chain can be cut off by using the pedigree chain marker sequence, so that multi-computer parallel processing can be carried out in the nodes, which reduces the computational overhead of iterative operation and effectively improves the performance of fault-tolerant computing.