ROOT’s RNTuple and the Case for Custom Scientific Data Formats

Each month, CERN collects and stores multiple petabytes of measurement data from the Large Hadron Collider (LHC) for further analysis. These data are stored in a custom columnar format provided by CERN’s open-source software framework ROOT. However, the increased availability of general-purpose columnar data formats raises the question whether developing and maintaining a custom data format for high-energy physics (HEP) is still worth it. To address this question, we compare ROOT’s RNTuple data format against Apache ORC and Apache Parquet. We find that RNTuple stores data 6-21% more efficiently, and processes physics data 3-6x faster.