Research on Heterogeneous Data Transfer Method for Power Energy Management

At present, the amount of data collected by electricity information acquisition system increases dramatically. For current data growth and business needs, based on the research results of Sqoop and DataX, which are cross-platform heterogeneous data migration tools based on big data platform, this paper proposes to adopt Spark technology to study the cross-platform heterogeneous data migration technology of big data acquisition platform (cloud platform) and Oracle database. Two example tests were conducted to verify the support and performance of the Spark migration tool for migration between data in different formats and the ability to read, parse, and support data in particular formats.