RIVAChain: Blockchain-based Integrity Verification for File Transfers

File transfer integrity verification is used to detect silent data corruption by calculating and comparing the checksum of files using secure hash functions, such as SHA-256. However, it incurs significant performance overhead due to I/O and compute-intensive checksum calculation process. In this paper, we present blockchain-based ledger architecture to store the checksum of frequently accessed scientific datasets to minimize the overhead of integrity verification. In the proposed architecture, the checksum of files is calculated and pushed to a private blockchain when they are first created such that future transfers will not require data source to recalculate checksum. As scientific datasets are typically generated in one location (e.g., observatory) and streamed to many geographically distributed locations to enable collaboration, eliminating checksum calculation for data sources will save a significant amount of resource consumption. Moreover, we find that blockchain-based integrity verification reduces transfer time by up to 50% when data source is the bottleneck in the integrity verification process. Finally, we show that private blockchains can scale to thousands of transactions per second thus they are better fit for scientific applications in which data generation rate can easily outpace the transaction confirmation rates of public blockchains.