Unlocking the Use of Raw Multispectral Earth Observation Imagery for Onboard Artificial Intelligence

Nowadays, there is growing interest in applying Artificial Intelligence (AI) on board Earth Observation (EO) satellites for time-critical applications, such as natural disaster response. However, the unavailability of raw satellite data currently hinders research of lightweight pre-processing techniques and limits the exploration of end-to-end pipelines, which extract insights directly from the source data. To fill this gap, this work presents a novel methodology to automate the creation of datasets for the detection of target events (e.g., warm thermal hotspots) or objects (e.g., vessels) from Sentinel-2 raw data and other multispectral EO pushbroom raw imagery. The presented approach first processes the raw data by applying a pipeline consisting of a spatial band registration and georeferencing of the raw data pixels. Then, it detects the target events by leveraging event-specific state-of-the-art algorithms on the Level-1C products, which are mosaicked and cropped on the georeferenced correspondent raw granule area. The detected events are, finally, re-projected back on the corresponding raw images. We apply the proposed methodology to realize THRawS (Thermal Hotspots in Raw Sentinel-2 data), the first dataset of Sentinel-2 raw data containing warm thermal hotspots. THRawS includes 1090 samples containing wildfires, volcanic eruptions, and 33335 eventfree acquisitions to enable thermal hotspot detection and general classification applications. This dataset and associated toolkits provide the community with both an immediately useful resource to speed up future research on energy-efficient pre-processing algorithms and AI-based end-to-end processing systems on board EO satellites.