Towards a Standard Archival Format for Earth Science Data: Storing NASA ECS Data using HDF5 Archival Information Packages (AIP)

Lack of standard metadata and file formats make it challenging to archive and distribute remote sensing data. Although the Common Data Model (CDM) developed by Unidata has the potential of providing easy access to data stored in a variety of file formats, it would still be necessary to maintain each I/O library ever used to create a data product. Moreover, due to the nature of these data structures, I/O libraries are often complicated and expensive to maintain. As a consequence, the long-term archival of data stored in multiple formats becomes an expensive and challenging task. In this paper, we demonstrate an alternative solution, namely migration of data to a single standards-based archive format. Data from NASA's Earth Observing System (EOS) Data Centers formatted in Hierarchical Data Format (HDF) are used in the demonstration.