BigDataCube: A Scalable, Federated Service Platform for Copernicus

The European Copernicus programme generates massive amounts of Earth Observation (EO) data, with the goal of improving our environmental understanding and management on local, regional, and global level. An initiative of this magnitude comes with a set of challenges, especially pertaining to effective service management once the huge volumes of raster data files are distributed across open data centers and commercial companies.The BigDataCube project responds to this challenge through the concept of federated, analysis-ready datacubes, exposed via the open OGC geo standard interfaces for interoperable access and processing, WMS, WCS, and WCPS.Such federated datacube services have been established in the project between the public German Copernicus hub, CODE-DE, and a commercial cloud provider, cloudeo AG, altogether offering access to more than 500 TB. Both data pools are federated in a location-transparent manner, establishing a common information space where users can query and combine datacubes without knowing their location of storage. In this nucleus of a growing federation recently the Alfred Wegener maritime research institute has joined, and further data centers are in the line. The spectrum of functionality available is showcased through several realistic use cases, including a value-adding Sentinel-1 SAR product on sea state and wind speed in the North Sea. The platform used is the pioneer datacube engine and OGC reference implementation, rasdaman. We report on the outcomes of the project, lessons learned, and further work foreseen in this area.