Large scale data storage and processing of insulator leakage current using HBase and mapreduce

In the smart grid environment, huge volumes of data will be accumulated from the condition monitoring system of power equipment. Using the traditional centralized storage architecture and relational databases, the performance of data querying and processing is slow, and cannot meet real-time requirements of the power equipment condition monitoring system. Meanwhile, MapReduce is a desirable parallel programming platform that is widely applied in kinds of data process fields. In this paper, a case-study on distributed storage using HBase and parallel processing of insulator leakage current data is presented. We propose efficient MapReduce based algorithms for parallel join query, parallel characteristics extraction and analogous assessment of insulator contamination degree. We evaluate our work on real large scale datasets utilizing Hadoop platform. Results reveal that the speedup and scale-up of our work are competent.